Wine Quality Dataset with custom layers- keras functional.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZL_6GK8qX35J"
   },
   "source": [
    "\n",
    "\n",
    "# Week 1: Multiple Output Models using the Keras Functional API\n",
    "\n",
    "Welcome to the first programming assignment of the course! Your task will be to use the Keras functional API to train a model to predict two outputs. For this lab, you will use the **[Wine Quality Dataset](https://archive.ics.uci.edu/ml/datasets/Wine+Quality)** from the **UCI machine learning repository**. It has separate datasets for red wine and white wine.\n",
    "\n",
    "Normally, the wines are classified into one of the quality ratings specified in the attributes. In this exercise, you will combine the two datasets to predict the wine quality and whether the wine is red or white solely from the attributes. \n",
    "\n",
    "You will model wine quality estimations as a regression problem and wine type detection as a binary classification problem.\n",
    "\n",
    "#### Please complete sections that are marked **(TODO)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "obdcD6urYBY9"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t8N3pcTQ5oQI"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.layers import Dense, Input\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import itertools\n",
    "\n",
    "import utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gQMERzWQYpgm"
   },
   "source": [
    "## Load Dataset\n",
    "\n",
    "\n",
    "You will now download the dataset from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php). \n",
    "\n",
    "### Pre-process the white wine dataset (TODO)\n",
    "You will add a new column named `is_red` in your dataframe to indicate if the wine is white or red. \n",
    "- In the white wine dataset, you will fill the column `is_red` with  zeros (0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "2qYAjKXCd4RH",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "cb9f284ba6255170fe04a309a407d72d",
     "grade": false,
     "grade_id": "cell-e5bfa0f152d9a21f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the white wine dataset\n",
    "URL = 'http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "white_df = pd.read_csv(URL, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with zeros.\n",
    "white_df[\"is_red\"] = 0# YOUR CODE HERE\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "white_df = white_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "defe38d6ec58fd31cd67b89e46c4373f",
     "grade": true,
     "grade_id": "cell-30575e713b55fc51",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "# You can click `File -> Open` in the menu above and open the `utils.py` file \n",
    "# in case you want to inspect the unit tests being used for each graded function.\n",
    "\n",
    "utils.test_white_df(white_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OQHK0ohBQRCk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.8\n",
      "9.1\n"
     ]
    }
   ],
   "source": [
    "print(white_df.alcohol[0])\n",
    "print(white_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 8.8\n",
    "# 9.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-process the red wine dataset (TODO)\n",
    "- In the red wine dataset, you will fill in the column `is_red` with ones (1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8y3QxKwBed8v",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "be72275f78e2c2d0038dde9aa63f8d4f",
     "grade": false,
     "grade_id": "cell-e47a40f306593274",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the red wine dataset\n",
    "URL = 'http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "red_df = pd.read_csv(URL, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with ones.\n",
    "red_df[\"is_red\"] =1 # YOUR CODE HERE\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "red_df = red_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d8e0c91b0fd668b63ba74a8f2f958b59",
     "grade": true,
     "grade_id": "cell-2a75937adcc0c25b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_red_df(red_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zsB3LUzNQpo_"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.2\n"
     ]
    }
   ],
   "source": [
    "print(red_df.alcohol[0])\n",
    "print(red_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2G8B-NYuM6-f"
   },
   "source": [
    "### Concatenate the datasets\n",
    "\n",
    "Next, concatenate the red and white wine dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YpQrOjJbfN3m"
   },
   "outputs": [],
   "source": [
    "df = pd.concat([red_df, white_df], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Se2dTmThQyjb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "9.5\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 9.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wx6y3rPpQv4k"
   },
   "outputs": [],
   "source": [
    "# NOTE: In a real-world scenario, you should shuffle the data.\n",
    "# YOU ARE NOT going to do that here because we want to test\n",
    "# with deterministic data. But if you want the code to do it,\n",
    "# it's in the commented line below:\n",
    "\n",
    "#df = df.iloc[np.random.permutation(len(df))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-EqIcbg5M_n1"
   },
   "source": [
    "This will chart the quality of the wines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IsvK0-Sgy17C"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQk0lEQVR4nO3df6zddX3H8edL6rAUmShyw1q2sqQxAs1QbhgbCbmMTasYwWUmJUxgc6khuOjWZCn7xy1LE5aM/ZANsg4cNSJNh5KSIU7CdudMRCyKKT8kdFKxlFEdiJQZtPjeH/eLXtrT9vbcH+fc83k+kpNz7ud8P9/zfvec+7rf+7nfc5qqQpLUhtcMugBJ0sIx9CWpIYa+JDXE0Jekhhj6ktSQJYMu4EhOOumkWrlyZV9zX3zxRZYtWza3BQ3IqPQyKn2AvQyrUelltn088MAD36uqNx84PvShv3LlSrZv397X3MnJSSYmJua2oAEZlV5GpQ+wl2E1Kr3Mto8k3+417vKOJDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZOjfkSsNqx1PPc+VG+7qa+6uay+a42qkmfFIX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JAjhn6SU5P8R5JHkzyc5CPd+BuT3JPk8e76xGlzrkmyM8ljSd45bfzsJDu6+z6eJPPTliSpl5kc6e8H1lfVW4FzgauTnA5sAO6tqlXAvd3XdPetBc4A1gA3JDmm29eNwDpgVXdZM4e9SJKO4IihX1VPV9XXutsvAI8Cy4GLgc3dZpuBS7rbFwNbquqlqnoC2Amck+QU4ISq+nJVFfDJaXMkSQvgqNb0k6wE3gZ8BRirqqdh6gcDcHK32XLgO9Om7e7Glne3DxyXJC2QJTPdMMnxwGeAj1bVDw6zHN/rjjrMeK/HWsfUMhBjY2NMTk7OtMxX2bdvX99zh82o9DIqfQCMLYX1q/f3NXfY/g1G6XkZlV7mq48ZhX6S1zIV+LdW1We74WeSnFJVT3dLN3u78d3AqdOmrwD2dOMreowfpKo2AZsAxsfHa2JiYmbdHGBycpJ+5w6bUellVPoAuP7WbVy3Y8bHTa+y67KJuS1mlkbpeRmVXuarj5mcvRPgZuDRqvrraXfdCVzR3b4C2DZtfG2SY5OcxtQfbO/vloBeSHJut8/Lp82RJC2AmRymnAd8ANiR5MFu7E+Ba4GtST4IPAm8H6CqHk6yFXiEqTN/rq6ql7t5VwG3AEuBu7uLJGmBHDH0q+pL9F6PB7jwEHM2Aht7jG8HzjyaAiVJc8d35EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JD+vtfnaUDrNxw14y2W796P1cesO2uay+aj5Ik9eCRviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG/uzJE8lebC7vHvafdck2ZnksSTvnDZ+dpId3X0fT5K5b0eSdDgzOdK/BVjTY/xvquqs7vI5gCSnA2uBM7o5NyQ5ptv+RmAdsKq79NqnJGkeHTH0q+qLwLMz3N/FwJaqeqmqngB2AuckOQU4oaq+XFUFfBK4pN+iJUn9WTKLuR9OcjmwHVhfVc8By4H7pm2zuxv7cXf7wPGekqxj6rcCxsbGmJyc7KvAffv29T132Ax7L+tX75/RdmNLD952mPs6nF69zNSw9Tzsr6+jMSq9zFcf/Yb+jcBfANVdXwf8PtBrnb4OM95TVW0CNgGMj4/XxMREX0VOTk7S79xhM+y9XLnhrhltt371fq7b8eqX3a7LJuahovl3/a3bDuplpoat52F/fR2NUellvvro6+ydqnqmql6uqp8A/wSc0921Gzh12qYrgD3d+Ioe45KkBdRX6Hdr9K94H/DKmT13AmuTHJvkNKb+YHt/VT0NvJDk3O6sncuBbbOoW5LUhyP+bprkNmACOCnJbuBjwESSs5haotkFfAigqh5OshV4BNgPXF1VL3e7uoqpM4GWAnd3F0nSAjpi6FfVpT2Gbz7M9huBjT3GtwNnHlV1kqQ55TtyJakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG3tjknuSPN5dnzjtvmuS7EzyWJJ3Ths/O8mO7r6PJ8nctyNJOpyZHOnfAqw5YGwDcG9VrQLu7b4myenAWuCMbs4NSY7p5twIrANWdZcD9ylJmmdHDP2q+iLw7AHDFwObu9ubgUumjW+pqpeq6glgJ3BOklOAE6rqy1VVwCenzZEkLZB+1/THquppgO765G58OfCdadvt7saWd7cPHJckLaAlc7y/Xuv0dZjx3jtJ1jG1FMTY2BiTk5N9FbNv376+5w6bYe9l/er9M9pubOnB2w5zX4fTq5eZGraeh/31dTRGpZf56qPf0H8mySlV9XS3dLO3G98NnDptuxXAnm58RY/xnqpqE7AJYHx8vCYmJvoqcnJykn7nDpth7+XKDXfNaLv1q/dz3Y5Xv+x2XTYxDxXNv+tv3XZQLzM1bD0P++vraIxKL/PVR7/LO3cCV3S3rwC2TRtfm+TYJKcx9Qfb+7sloBeSnNudtXP5tDmSpAVyxMOUJLcBE8BJSXYDHwOuBbYm+SDwJPB+gKp6OMlW4BFgP3B1Vb3c7eoqps4EWgrc3V0kSQvoiKFfVZce4q4LD7H9RmBjj/HtwJlHVZ0kaU75jlxJaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhoy15+9I2nIrezxkRnrV++f0Udp7Lr2ovkoSQvII31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZFahn2RXkh1JHkyyvRt7Y5J7kjzeXZ84bftrkuxM8liSd862eEnS0ZmLI/0Lquqsqhrvvt4A3FtVq4B7u69JcjqwFjgDWAPckOSYOXh8SdIMzcfyzsXA5u72ZuCSaeNbquqlqnoC2AmcMw+PL0k6hFRV/5OTJ4DngAL+sao2Jfl+Vb1h2jbPVdWJSf4euK+qPtWN3wzcXVW399jvOmAdwNjY2Nlbtmzpq759+/Zx/PHH9zV32Ax7Lzueen5G240thWd++Oqx1ct/fh4qmn97n33+oF5mapA993quej0vvSyG52rYv1dmarZ9XHDBBQ9MW4H5qSWzqgrOq6o9SU4G7knyzcNsmx5jPX/iVNUmYBPA+Ph4TUxM9FXc5OQk/c4dNsPey5Ub7prRdutX7+e6Ha9+2e26bGIeKpp/19+67aBeZmqQPfd6rno9L70shudq2L9XZmq++pjV8k5V7emu9wJ3MLVc80ySUwC6673d5ruBU6dNXwHsmc3jS5KOTt+hn2RZkte/cht4B/AQcCdwRbfZFcC27vadwNokxyY5DVgF3N/v40uSjt5slnfGgDuSvLKfT1fV55N8Fdia5IPAk8D7Aarq4SRbgUeA/cDVVfXyrKqXJB2VvkO/qr4F/EqP8f8FLjzEnI3Axn4fU5I0O74jV5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDen7P0bX/Fm54a6Dxtav3s+VPcYPtOvai+ajJEkjwiN9SWqIoS9JDTH0Jakhhr4kNcTQl6SGePaOpKHX64y2Q+l1pptntf2MR/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQBX9HbpI1wN8BxwA3VdW18/VYO556fkafQd+L7+CTNIoW9Eg/yTHAPwDvAk4HLk1y+kLWIEktW+gj/XOAnVX1LYAkW4CLgUcWuA5JmndH85lBB7plzbI5rORnUlXzsuOeD5b8DrCmqv6g+/oDwK9W1YcP2G4dsK778i3AY30+5EnA9/qcO2xGpZdR6QPsZViNSi+z7eOXqurNBw4u9JF+eowd9FOnqjYBm2b9YMn2qhqf7X6Gwaj0Mip9gL0Mq1HpZb76WOizd3YDp077egWwZ4FrkKRmLXTofxVYleS0JD8HrAXuXOAaJKlZC7q8U1X7k3wY+DemTtn8RFU9PI8POesloiEyKr2MSh9gL8NqVHqZlz4W9A+5kqTB8h25ktQQQ1+SGjJyoZ/kdUnuT/KNJA8n+fNB1zRbSY5J8vUk/zroWmYjya4kO5I8mGT7oOuZjSRvSHJ7km8meTTJrw26pqOV5C3dc/HK5QdJPjrouvqV5I+67/mHktyW5HWDrqlfST7S9fHwXD8nI7emnyTAsqral+S1wJeAj1TVfQMurW9J/hgYB06oqvcMup5+JdkFjFfVon/jTJLNwH9V1U3dmWjHVdX3B11Xv7qPSHmKqTdLfnvQ9RytJMuZ+l4/vap+mGQr8LmqumWwlR29JGcCW5j6BIMfAZ8Hrqqqx+di/yN3pF9T9nVfvra7LNqfbElWABcBNw26Fk1JcgJwPnAzQFX9aDEHfudC4L8XY+BPswRYmmQJcByL9z1AbwXuq6r/q6r9wH8C75urnY9c6MNPl0MeBPYC91TVVwZd0yz8LfAnwE8GXcgcKOALSR7oPmpjsfpl4LvAP3fLbjclmZ8PSlk4a4HbBl1Ev6rqKeCvgCeBp4Hnq+oLg62qbw8B5yd5U5LjgHfz6je1zspIhn5VvVxVZzH1jt9zul+XFp0k7wH2VtUDg65ljpxXVW9n6lNWr05y/qAL6tMS4O3AjVX1NuBFYMNgS+pftzz1XuBfBl1Lv5KcyNSHN54G/AKwLMnvDraq/lTVo8BfAvcwtbTzDWD/XO1/JEP/Fd2v3JPAmgGX0q/zgPd2a+FbgN9I8qnBltS/qtrTXe8F7mBqzXIx2g3snvYb5O1M/RBYrN4FfK2qnhl0IbPwm8ATVfXdqvox8Fng1wdcU9+q6uaqentVnQ88C8zJej6MYOgneXOSN3S3lzL1YvjmYKvqT1VdU1UrqmolU79+/3tVLcqjlyTLkrz+ldvAO5j6NXbRqar/Ab6T5C3d0IUs7o8Hv5RFvLTTeRI4N8lx3ckcFwKPDrimviU5ubv+ReC3mcPnZ8H/56wFcAqwuTsb4TXA1qpa1Kc6jogx4I6p70eWAJ+uqs8PtqRZ+UPg1m5p5FvA7w24nr50a8a/BXxo0LXMRlV9JcntwNeYWgr5Oov74xg+k+RNwI+Bq6vqubna8cidsilJOrSRW96RJB2aoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8v9wPc08ifqwpQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Nut1rmYLzf-p"
   },
   "source": [
    "### Imbalanced data (TODO)\n",
    "You can see from the plot above that the wine quality dataset is imbalanced. \n",
    "- Since there are very few observations with quality equal to 3, 4, 8 and 9, you can drop these observations from your dataset. \n",
    "- You can do this by removing data belonging to all classes except those > 4 and < 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "doH9_-gnf3sz",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d9ba9fc3a3ca02ccc567be33652b80fe",
     "grade": false,
     "grade_id": "cell-6a3e9db696f6827b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "# get data with wine quality greater than 4 and less than 8\n",
    "df = df[(df['quality'] > 4) & (df['quality'] < 8 )]\n",
    "\n",
    "# reset index and drop the old one\n",
    "df = df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "281e1d86a4803560ed5892cd7eda4c01",
     "grade": true,
     "grade_id": "cell-aed3da719d4682c7",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_df_drop(df)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xNR1iAlMRPXO"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.9\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cwhuRpnVRTzG"
   },
   "source": [
    "You can plot again to see the new range of data and quality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "857ygzZiLgGg"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQaUlEQVR4nO3df6zddX3H8edrRQkW+ZXOO9IywaQx48dE2zDUzN2GRapuK/vDpIQIZCydBpOZkEWYyTRZmuAfbAk4yDpxQGQ2zB8rEdlGGDdmU8TiwPJDtEonpUinIFBiMLD3/jjfbsfLae85p/ec2/p5PpKT8z2f7/fz/b6/Xz687rmfe863qSokSW34laUuQJI0PYa+JDXE0Jekhhj6ktQQQ1+SGnLUUhewkBUrVtSpp546Vt8XX3yR5cuXL25Bi8C6RmNdo7Gu0fyy1nX//ff/uKp+9VUrquqwfqxZs6bGdc8994zdd5KsazTWNRrrGs0va13A9hqQqU7vSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQw772zBIh6sdTz7HpVfeMVbfXVe/b5GrkYbjO31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDFgz9JKckuSfJo0keTvKnXftJSe5K8r3u+cS+Plcl2ZnksSTn97WvSbKjW3dtkkzmtCRJgwzzTv9l4Iqq+g3gXODyJKcDVwJ3V9Vq4O7uNd26jcAZwHrg+iTLun3dAGwCVneP9Yt4LpKkBSwY+lX1VFV9q1t+AXgUWAlsAG7uNrsZuKBb3gBsraqXqupxYCdwTpKTgeOq6utVVcAtfX0kSVMw0px+klOBtwLfAGaq6ino/WAA3tBtthJ4oq/b7q5tZbc8v12SNCVHDbthkmOBLwAfqarnDzIdP2hFHaR90LE20ZsGYmZmhrm5uWHL/AX79u0bu+8kWddoDte6Zo6BK856eay+kzyfw/V6WddoJlXXUKGf5DX0Av/Wqvpi1/x0kpOr6qlu6mZv174bOKWv+ypgT9e+akD7q1TVFmALwNq1a2t2dna4s5lnbm6OcftOknWN5nCt67pbt3HNjqHfN/2CXRfNLm4xfQ7X62Vdo5lUXcN8eifAjcCjVfVXfatuBy7pli8BtvW1b0xydJLT6P3B9r5uCuiFJOd2+7y4r48kaQqGeZvyTuADwI4kD3Rtfw5cDdyW5DLgh8D7Aarq4SS3AY/Q++TP5VX1StfvQ8BNwDHAnd1DkjQlC4Z+Vf07g+fjAc47QJ/NwOYB7duBM0cpUJK0ePxGriQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDVkvH/V+Qix48nnuPTKO8bqu+vq9y1yNZK09HynL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIQuGfpLPJNmb5KG+tk8keTLJA93jvX3rrkqyM8ljSc7va1+TZEe37tokWfzTkSQdzDDv9G8C1g9o/+uqOrt7fAUgyenARuCMrs/1SZZ1298AbAJWd49B+5QkTdCCoV9VXwWeGXJ/G4CtVfVSVT0O7ATOSXIycFxVfb2qCrgFuGDcoiVJ40kvgxfYKDkV+HJVndm9/gRwKfA8sB24oqqeTfIp4N6q+my33Y3AncAu4Oqq+t2u/beBj1bV7x3geJvo/VbAzMzMmq1bt451cnufeY6nfzZWV85aefx4HYewb98+jj322Intf1zWNRrH12isazSHWte6devur6q189uPGnN/NwB/CVT3fA3wR8Cgefo6SPtAVbUF2AKwdu3amp2dHavI627dxjU7xjvFXReNd8xhzM3NMe45TZJ1jcbxNRrrGs2k6hrr0ztV9XRVvVJV/wP8HXBOt2o3cErfpquAPV37qgHtkqQpGiv0uzn6/f4Q2P/JntuBjUmOTnIavT/Y3ldVTwEvJDm3+9TOxcC2Q6hbkjSGBX83TfI5YBZYkWQ38HFgNsnZ9KZodgF/AlBVDye5DXgEeBm4vKpe6Xb1IXqfBDqG3jz/nYt5IpKkhS0Y+lV14YDmGw+y/WZg84D27cCZI1UnSVpUfiNXkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyIKhn+QzSfYmeaiv7aQkdyX5Xvd8Yt+6q5LsTPJYkvP72tck2dGtuzZJFv90JEkHM8w7/ZuA9fPargTurqrVwN3da5KcDmwEzuj6XJ9kWdfnBmATsLp7zN+nJGnCFgz9qvoq8My85g3Azd3yzcAFfe1bq+qlqnoc2Amck+Rk4Liq+npVFXBLXx9J0pSMO6c/U1VPAXTPb+jaVwJP9G23u2tb2S3Pb5ckTdFRi7y/QfP0dZD2wTtJNtGbCmJmZoa5ubmxipk5Bq446+Wx+o57zGHs27dvovsfl3WNxvE1GusazaTqGjf0n05yclU91U3d7O3adwOn9G23CtjTta8a0D5QVW0BtgCsXbu2Zmdnxyryulu3cc2O8U5x10XjHXMYc3NzjHtOk2Rdo3F8jca6RjOpusad3rkduKRbvgTY1te+McnRSU6j9wfb+7opoBeSnNt9aufivj6SpClZ8G1Kks8Bs8CKJLuBjwNXA7cluQz4IfB+gKp6OMltwCPAy8DlVfVKt6sP0fsk0DHAnd1DkjRFC4Z+VV14gFXnHWD7zcDmAe3bgTNHqk6StKj8Rq4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ1Z7HvvSJI6p155x9h9b1q/fBEr+X++05ekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDin0k+xKsiPJA0m2d20nJbkryfe65xP7tr8qyc4kjyU5/1CLlySNZjHe6a+rqrOram33+krg7qpaDdzdvSbJ6cBG4AxgPXB9kmWLcHxJ0pAmMb2zAbi5W74ZuKCvfWtVvVRVjwM7gXMmcHxJ0gGkqsbvnDwOPAsU8LdVtSXJT6vqhL5tnq2qE5N8Cri3qj7btd8I3FlVnx+w303AJoCZmZk1W7duHau+vc88x9M/G6srZ608fryOQ9i3bx/HHnvsxPY/LusajeNrNC3WtePJ58bue9rxyw6prnXr1t3fNwPzf44ae48976yqPUneANyV5DsH2TYD2gb+xKmqLcAWgLVr19bs7OxYxV136zau2THeKe66aLxjDmNubo5xz2mSrGs0jq/RtFjXpVfeMXbfm9Yvn0hdhzS9U1V7uue9wJfoTdc8neRkgO55b7f5buCUvu6rgD2HcnxJ0mjGDv0ky5O8fv8y8G7gIeB24JJus0uAbd3y7cDGJEcnOQ1YDdw37vElSaM7lOmdGeBLSfbv5x+q6p+TfBO4LcllwA+B9wNU1cNJbgMeAV4GLq+qVw6peknSSMYO/ar6AfCWAe0/Ac47QJ/NwOZxjylJOjR+I1eSGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDZl66CdZn+SxJDuTXDnt40tSy6Ya+kmWAX8DvAc4HbgwyenTrEGSWjbtd/rnADur6gdV9XNgK7BhyjVIUrOOmvLxVgJP9L3eDfzW/I2SbAI2dS/3JXlszOOtAH48Tsd8cswjDmfsuibMukbj+BqNdY1g3ScPua43DmqcduhnQFu9qqFqC7DlkA+WbK+qtYe6n8VmXaOxrtFY12haq2va0zu7gVP6Xq8C9ky5Bklq1rRD/5vA6iSnJXktsBG4fco1SFKzpjq9U1UvJ/kw8C/AMuAzVfXwBA95yFNEE2Jdo7Gu0VjXaJqqK1WvmlKXJP2S8hu5ktQQQ1+SGnLEhn6SXUl2JHkgyfYB65Pk2u52D99O8ra+dRO7FcQQdV3U1fPtJF9L8pZh+064rtkkz3XrH0jyF33rlvJ6/VlfTQ8leSXJScP0PcS6Tkjy+STfSfJokrfPW79U42uhupZqfC1U11KNr4Xqmvr4SvLmvmM+kOT5JB+Zt83kxldVHZEPYBew4iDr3wvcSe+7AecC3+jalwHfB94EvBZ4EDh9inW9AzixW37P/rqG6TvhumaBLw9oX9LrNW/b3wf+bUrX62bgj7vl1wInHCbja6G6lmp8LVTXUo2vg9a1VONr3vn/CHjjtMbXEftOfwgbgFuq517ghCQns8S3gqiqr1XVs93Le+l9V+FwdjjdOuNC4HOTPkiS44B3ATcCVNXPq+qn8zab+vgapq6lGF9DXq8DWdLrNc9Uxtc85wHfr6r/mtc+sfF1JId+Af+a5P70btsw36BbPqw8SPu06up3Gb2f5uP0nURdb0/yYJI7k5zRtR0W1yvJ64D1wBdG7TuGNwH/Dfx9kv9M8ukky+dtsxTja5i6+k1rfA1b17TH19DXa8rjq99GBv+gmdj4OpJD/51V9TZ6v8JenuRd89Yf6JYPQ90KYoJ19YpL1tH7n/Kjo/adUF3fovcr5luA64B/2l/qgH1N/XrR+9X7P6rqmTH6juoo4G3ADVX1VuBFYP7c6VKMr2Hq6hU33fE1TF1LMb6Gvl5Md3wBkN4XVP8A+MdBqwe0Lcr4OmJDv6r2dM97gS/R+7Wn34Fu+TDRW0EMURdJfhP4NLChqn4ySt9J1VVVz1fVvm75K8BrkqzgMLhenVe9I5rg9doN7K6qb3SvP08vPOZvM+3xNUxdSzG+FqxricbXUNerM83xtd97gG9V1dMD1k1sfB2RoZ9keZLX718G3g08NG+z24GLu7+Cnws8V1VPMcFbQQxTV5JfB74IfKCqvjviOU2yrl9Lkm75HHpj4ycs8fXq1h0P/A6wbdS+46iqHwFPJHlz13Qe8Mi8zaY+voapaynG15B1TX18Dfnfcerjq8/B/oYwufG1WH+FnuaD3lzdg93jYeBjXfsHgQ92y6H3D7Z8H9gBrO3r/17gu926j025rk8DzwIPdI/tB+s7xbo+3K17kN4fAN9xOFyv7vWlwNZh+i5ibWcD24Fv05uKOHGpx9eQdU19fA1Z19TH1zB1LeH4eh29H3rH97VNZXx5GwZJasgROb0jSRqPoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8r889HO6f1QuqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n3k0vqSsp84t"
   },
   "source": [
    "### Train Test Split (TODO)\n",
    "\n",
    "Next, you can split the datasets into training, test and validation datasets.\n",
    "- The data frame should be split 80:20 into `train` and `test` sets.\n",
    "- The resulting `train` should then be split 80:20 into `train` and `val` sets.\n",
    "- The `train_test_split` parameter `test_size` takes a float value that ranges between 0. and 1, and represents the proportion of the dataset that is allocated to the test set.  The rest of the data is allocated to the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "PAVIf2-fgRVY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7f5738f4fb51d65adc9a8acbdf2b9970",
     "grade": false,
     "grade_id": "cell-91946cadf745206b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# Please do not change the random_state parameter. This is needed for grading.\n",
    "\n",
    "# split df into 80:20 train and test sets\n",
    "train, test = train_test_split(df, test_size=0.2, random_state = 1)\n",
    "                               \n",
    "# split train into 80:20 train and val sets\n",
    "train, val = train_test_split(train, test_size=0.2, random_state = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "editable": false,
    "id": "57h9LcEzRWpk",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "42adbe9e66efac7c7a5f8cd73ac92f22",
     "grade": true,
     "grade_id": "cell-64b8b38cd0b965f6",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_data_sizes(train.size, test.size, val.size)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RwTNu4KFqG-K"
   },
   "source": [
    "Here's where you can explore the training stats. You can pop the labels 'is_red' and 'quality' from the data as these will be used as the labels\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y_afyhhHM6WQ"
   },
   "outputs": [],
   "source": [
    "train_stats = train.describe()\n",
    "train_stats.pop('is_red')\n",
    "train_stats.pop('quality')\n",
    "train_stats = train_stats.transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ahvbYm4fNqSt"
   },
   "source": [
    "Explore the training stats!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n_gAtPjZ0otF"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fixed acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>7.221616</td>\n",
       "      <td>1.325297</td>\n",
       "      <td>3.80000</td>\n",
       "      <td>6.40000</td>\n",
       "      <td>7.00000</td>\n",
       "      <td>7.7000</td>\n",
       "      <td>15.60000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>volatile acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.338929</td>\n",
       "      <td>0.162476</td>\n",
       "      <td>0.08000</td>\n",
       "      <td>0.23000</td>\n",
       "      <td>0.29000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>citric acid</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.321569</td>\n",
       "      <td>0.147970</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.25000</td>\n",
       "      <td>0.31000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.66000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residual sugar</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>5.155911</td>\n",
       "      <td>4.639632</td>\n",
       "      <td>0.60000</td>\n",
       "      <td>1.80000</td>\n",
       "      <td>2.80000</td>\n",
       "      <td>7.6500</td>\n",
       "      <td>65.80000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chlorides</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.056976</td>\n",
       "      <td>0.036802</td>\n",
       "      <td>0.01200</td>\n",
       "      <td>0.03800</td>\n",
       "      <td>0.04700</td>\n",
       "      <td>0.0660</td>\n",
       "      <td>0.61100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>30.388590</td>\n",
       "      <td>17.236784</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>17.00000</td>\n",
       "      <td>28.00000</td>\n",
       "      <td>41.0000</td>\n",
       "      <td>131.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>115.062282</td>\n",
       "      <td>56.706617</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>75.00000</td>\n",
       "      <td>117.00000</td>\n",
       "      <td>156.0000</td>\n",
       "      <td>344.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>density</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.994633</td>\n",
       "      <td>0.003005</td>\n",
       "      <td>0.98711</td>\n",
       "      <td>0.99232</td>\n",
       "      <td>0.99481</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>1.03898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pH</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>3.223201</td>\n",
       "      <td>0.161272</td>\n",
       "      <td>2.72000</td>\n",
       "      <td>3.11000</td>\n",
       "      <td>3.21000</td>\n",
       "      <td>3.3300</td>\n",
       "      <td>4.01000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sulphates</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.534051</td>\n",
       "      <td>0.149149</td>\n",
       "      <td>0.22000</td>\n",
       "      <td>0.43000</td>\n",
       "      <td>0.51000</td>\n",
       "      <td>0.6000</td>\n",
       "      <td>1.95000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>alcohol</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>10.504466</td>\n",
       "      <td>1.154654</td>\n",
       "      <td>8.50000</td>\n",
       "      <td>9.50000</td>\n",
       "      <td>10.30000</td>\n",
       "      <td>11.3000</td>\n",
       "      <td>14.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       count        mean        std      min       25%  \\\n",
       "fixed acidity         3155.0    7.221616   1.325297  3.80000   6.40000   \n",
       "volatile acidity      3155.0    0.338929   0.162476  0.08000   0.23000   \n",
       "citric acid           3155.0    0.321569   0.147970  0.00000   0.25000   \n",
       "residual sugar        3155.0    5.155911   4.639632  0.60000   1.80000   \n",
       "chlorides             3155.0    0.056976   0.036802  0.01200   0.03800   \n",
       "free sulfur dioxide   3155.0   30.388590  17.236784  1.00000  17.00000   \n",
       "total sulfur dioxide  3155.0  115.062282  56.706617  6.00000  75.00000   \n",
       "density               3155.0    0.994633   0.003005  0.98711   0.99232   \n",
       "pH                    3155.0    3.223201   0.161272  2.72000   3.11000   \n",
       "sulphates             3155.0    0.534051   0.149149  0.22000   0.43000   \n",
       "alcohol               3155.0   10.504466   1.154654  8.50000   9.50000   \n",
       "\n",
       "                            50%       75%        max  \n",
       "fixed acidity           7.00000    7.7000   15.60000  \n",
       "volatile acidity        0.29000    0.4000    1.24000  \n",
       "citric acid             0.31000    0.4000    1.66000  \n",
       "residual sugar          2.80000    7.6500   65.80000  \n",
       "chlorides               0.04700    0.0660    0.61100  \n",
       "free sulfur dioxide    28.00000   41.0000  131.00000  \n",
       "total sulfur dioxide  117.00000  156.0000  344.00000  \n",
       "density                 0.99481    0.9968    1.03898  \n",
       "pH                      3.21000    3.3300    4.01000  \n",
       "sulphates               0.51000    0.6000    1.95000  \n",
       "alcohol                10.30000   11.3000   14.00000  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bGPvt9jir_HC"
   },
   "source": [
    "### Get the labels (TODO)\n",
    "\n",
    "The features and labels are currently in the same dataframe.\n",
    "- You will want to store the label columns `is_red` and `quality` separately from the feature columns.  \n",
    "- The following function, `format_output`, gets these two columns from the dataframe (it's given to you).\n",
    "- `format_output` also formats the data into numpy arrays. \n",
    "- Please use the `format_output` and apply it to the `train`, `val` and `test` sets to get dataframes for the labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z_fs14XQqZVP"
   },
   "outputs": [],
   "source": [
    "def format_output(data):\n",
    "    is_red = data.pop('is_red')\n",
    "    is_red = np.array(is_red)\n",
    "    quality = data.pop('quality')\n",
    "    quality = np.array(quality)\n",
    "    return (quality, is_red)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8L3ZZe1fQicm",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7a86809e54895a816434c48dc903f55d",
     "grade": false,
     "grade_id": "cell-5c30fa2c2a354b0f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# format the output of the train set\n",
    "train_Y = format_output(train)# YOUR CODE HERE)\n",
    "\n",
    "# format the output of the val set\n",
    "val_Y = format_output(val)# YOUR CODE HERE)\n",
    "    \n",
    "# format the output of the test set\n",
    "test_Y = format_output(test)# YOUR CODE HERE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "359cabbafaed14ec9bbc1e57a7b6f32c",
     "grade": true,
     "grade_id": "cell-4977d8befb80f56b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_format_output(df, train_Y, val_Y, test_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that after you get the labels, the `train`, `val` and `test` dataframes no longer contain the label columns, and contain just the feature columns.\n",
    "- This is because you used `.pop` in the `format_output` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>7.5</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.18</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.088</td>\n",
       "      <td>27.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>0.99915</td>\n",
       "      <td>3.38</td>\n",
       "      <td>0.77</td>\n",
       "      <td>9.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3557</th>\n",
       "      <td>6.3</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.29</td>\n",
       "      <td>12.2</td>\n",
       "      <td>0.044</td>\n",
       "      <td>59.0</td>\n",
       "      <td>196.0</td>\n",
       "      <td>0.99782</td>\n",
       "      <td>3.14</td>\n",
       "      <td>0.40</td>\n",
       "      <td>8.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3825</th>\n",
       "      <td>8.8</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.25</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.024</td>\n",
       "      <td>52.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>0.99250</td>\n",
       "      <td>2.87</td>\n",
       "      <td>0.49</td>\n",
       "      <td>11.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1740</th>\n",
       "      <td>6.4</td>\n",
       "      <td>0.45</td>\n",
       "      <td>0.07</td>\n",
       "      <td>1.1</td>\n",
       "      <td>0.030</td>\n",
       "      <td>10.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>0.99050</td>\n",
       "      <td>2.97</td>\n",
       "      <td>0.28</td>\n",
       "      <td>10.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1221</th>\n",
       "      <td>7.2</td>\n",
       "      <td>0.53</td>\n",
       "      <td>0.13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.058</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.99573</td>\n",
       "      <td>3.21</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "225             7.5              0.65         0.18             7.0      0.088   \n",
       "3557            6.3              0.27         0.29            12.2      0.044   \n",
       "3825            8.8              0.27         0.25             5.0      0.024   \n",
       "1740            6.4              0.45         0.07             1.1      0.030   \n",
       "1221            7.2              0.53         0.13             2.0      0.058   \n",
       "\n",
       "      free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "225                  27.0                  94.0  0.99915  3.38       0.77   \n",
       "3557                 59.0                 196.0  0.99782  3.14       0.40   \n",
       "3825                 52.0                  99.0  0.99250  2.87       0.49   \n",
       "1740                 10.0                 131.0  0.99050  2.97       0.28   \n",
       "1221                 18.0                  22.0  0.99573  3.21       0.68   \n",
       "\n",
       "      alcohol  \n",
       "225       9.4  \n",
       "3557      8.8  \n",
       "3825     11.4  \n",
       "1740     10.8  \n",
       "1221      9.9  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hEdbrruAsN1D"
   },
   "source": [
    "### Normalize the data (TODO)\n",
    "\n",
    "Next, you can normalize the data, x, using the formula:\n",
    "$$x_{norm} = \\frac{x - \\mu}{\\sigma}$$\n",
    "- The `norm` function is defined for you.\n",
    "- Please apply the `norm` function to normalize the dataframes that contains the feature columns of `train`, `val` and `test` sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WWiZPAHCLjUs"
   },
   "outputs": [],
   "source": [
    "def norm(x):\n",
    "    return (x - train_stats['mean']) / train_stats['std']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "JEaOi2I2Lk69",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6bc0cdcb563d192f271067aa3373ff32",
     "grade": false,
     "grade_id": "cell-d8416d975c371095",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# normalize the train set\n",
    "norm_train_X = norm(train)# YOUR CODE HERE)\n",
    "    \n",
    "# normalize the val set\n",
    "norm_val_X = norm(val)# YOUR CODE HERE)\n",
    "    \n",
    "# normalize the test set\n",
    "norm_test_X = norm(test)# YOUR CODE HERE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4f567db45bf40191601780379cc100b8",
     "grade": true,
     "grade_id": "cell-97fad979d157529b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_norm(norm_train_X, norm_val_X, norm_test_X, train, val, test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hzykDwQhsaPO"
   },
   "source": [
    "## Define the Model (TODO)\n",
    "\n",
    "Define the model using the functional API. The base model will be 2 `Dense` layers of 128 neurons each, and have the `'relu'` activation.\n",
    "- Check out the documentation for [tf.keras.layers.Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "Rhcns3oTFkM6",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "74b031247e569526552bf13a034a1c07",
     "grade": false,
     "grade_id": "cell-73fceedad1fe351c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def base_model(inputs):\n",
    "    \n",
    "    # connect a Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation='relu')(inputs) # YOUR CODE HERE\n",
    "    \n",
    "    # connect another Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation='relu')(x)# YOUR CODE HERE\n",
    "    return x\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9255924b3def80f679616e4c851a43e1",
     "grade": true,
     "grade_id": "cell-54f742a133353d75",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_base_model(base_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xem_fcVws6Kz"
   },
   "source": [
    "# Define output layers of the model (TODO)\n",
    "\n",
    "You will add output layers to the base model. \n",
    "- The model will need two outputs.\n",
    "\n",
    "One output layer will predict wine quality, which is a numeric value.\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since this is a regression output, the activation can be left as its default value `None`.\n",
    "\n",
    "The other output layer will predict the wine type, which is either red `1` or not red `0` (white).\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since there are two possible categories, you can use a sigmoid activation for binary classification.\n",
    "\n",
    "Define the `Model`\n",
    "- Define the `Model` object, and set the following parameters:\n",
    "  - `inputs`: pass in the inputs to the model as a list.\n",
    "  - `outputs`: pass in a list of the outputs that you just defined: wine quality, then wine type.\n",
    "  - **Note**: please list the wine quality before wine type in the outputs, as this will affect the calculated loss if you choose the other order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "n5UGF8PMVLPt",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "76d35b90d20cdcbb22986cd8211057de",
     "grade": false,
     "grade_id": "cell-19e285f482f021fb",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def final_model(inputs):\n",
    "    \n",
    "    # get the base model\n",
    "    x = base_model(inputs)\n",
    "\n",
    "    # connect the output Dense layer for regression\n",
    "    wine_quality = Dense(units='1', name='wine_quality')(x)\n",
    "\n",
    "    # connect the output Dense layer for classification. this will use a sigmoid activation.\n",
    "    wine_type = Dense(units='1', activation='sigmoid', name='wine_type')(x)\n",
    "\n",
    "    # define the model using the input and output layers\n",
    "    model = Model(inputs=inputs, outputs=[wine_quality, wine_type])# YOUR CODE HERE)\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "89cbf89d8ab5e2e59ecf7f63f517520a",
     "grade": true,
     "grade_id": "cell-40d050f855c817d1",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_final_model(final_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5R0BMTsltZyu"
   },
   "source": [
    "## Compiling the Model\n",
    "\n",
    "Next, compile the model. When setting the loss parameter of `model.compile`, you're setting the loss for each of the two outputs (wine quality and wine type).\n",
    "\n",
    "To set more than one loss, use a dictionary of key-value pairs.\n",
    "- You can look at the docs for the losses [here](https://www.tensorflow.org/api_docs/python/tf/keras/losses#functions).\n",
    "    - **Note**: For the desired spelling, please look at the \"Functions\" section of the documentation and not the \"classes\" section on that same page.\n",
    "- wine_type: Since you will be performing binary classification on wine type, you should use the binary crossentropy loss function for it.  Please pass this in as a string.  \n",
    "  - **Hint**, this should be all lowercase.  In the documentation, you'll see this under the \"Functions\" section, not the \"Classes\" section.\n",
    "- wine_quality: since this is a regression output, use the mean squared error.  Please pass it in as a string, all lowercase.\n",
    "  - **Hint**: You may notice that there are two aliases for mean squared error.  Please use the shorter name.\n",
    "\n",
    "\n",
    "You will also set the metric for each of the two outputs.  Again, to set metrics for two or more outputs, use a dictionary with key value pairs.\n",
    "- The metrics documentation is linked [here](https://www.tensorflow.org/api_docs/python/tf/keras/metrics).\n",
    "- For the wine type, please set it to accuracy as a string, all lowercase.\n",
    "- For wine quality, please use the root mean squared error.  Instead of a string, you'll set it to an instance of the class [RootMeanSquaredError](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/RootMeanSquaredError), which belongs to the tf.keras.metrics module.\n",
    "\n",
    "**Note**: If you see the error message \n",
    ">Exception: wine quality loss function is incorrect.\n",
    "\n",
    "- Please also check your other losses and metrics, as the error may be caused by the other three key-value pairs and not the wine quality loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "LK11duUbUjmh",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "22f45067ca69eb2ccadb43874dbcc27b",
     "grade": false,
     "grade_id": "cell-81afdc4dcca51d5e",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "inputs = tf.keras.layers.Input(shape=(11,))\n",
    "rms = tf.keras.optimizers.RMSprop(lr=0.0001)\n",
    "model = final_model(inputs)\n",
    "\n",
    "model.compile(optimizer=rms, \n",
    "              loss = {'wine_type' : 'binary_crossentropy',\n",
    "                      'wine_quality' : 'mse' # YOUR CODE HERE\n",
    "                     },\n",
    "              metrics = {'wine_type' : 'accuracy',\n",
    "                         'wine_quality':  tf.keras.metrics.RootMeanSquaredError()# YOUR CODE HERE\n",
    "                       }\n",
    "             )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "88e02238ea5e456ff65e835cc8158054",
     "grade": true,
     "grade_id": "cell-2eeeba02391c4632",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_model_compile(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "90MpAMpWuKm-"
   },
   "source": [
    "## Training the Model\n",
    "\n",
    "Fit the model to the training inputs and outputs. \n",
    "- Check the documentation for [model.fit](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit).\n",
    "- Remember to use the normalized training set as inputs. \n",
    "- For the validation data, please use the normalized validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "_eiZkle4XwiY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2ca7664be03bfd6fd3651ae44d17b793",
     "grade": false,
     "grade_id": "cell-0bb56262896f6680",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3155 samples, validate on 789 samples\n",
      "Epoch 1/180\n",
      "3155/3155 [==============================] - 1s 364us/sample - loss: 23.8315 - wine_quality_loss: 23.1535 - wine_type_loss: 0.6419 - wine_quality_root_mean_squared_error: 4.8155 - wine_type_accuracy: 0.7645 - val_loss: 16.1527 - val_wine_quality_loss: 15.5373 - val_wine_type_loss: 0.6269 - val_wine_quality_root_mean_squared_error: 3.9402 - val_wine_type_accuracy: 0.7883\n",
      "Epoch 2/180\n",
      "3155/3155 [==============================] - 0s 94us/sample - loss: 10.3616 - wine_quality_loss: 9.7479 - wine_type_loss: 0.6006 - wine_quality_root_mean_squared_error: 3.1243 - wine_type_accuracy: 0.8120 - val_loss: 5.7017 - val_wine_quality_loss: 5.1851 - val_wine_type_loss: 0.5611 - val_wine_quality_root_mean_squared_error: 2.2672 - val_wine_type_accuracy: 0.8390\n",
      "Epoch 3/180\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 3.8686 - wine_quality_loss: 3.3541 - wine_type_loss: 0.5063 - wine_quality_root_mean_squared_error: 1.8335 - wine_type_accuracy: 0.8415 - val_loss: 2.7705 - val_wine_quality_loss: 2.3715 - val_wine_type_loss: 0.4429 - val_wine_quality_root_mean_squared_error: 1.5256 - val_wine_type_accuracy: 0.8758\n",
      "Epoch 4/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 2.5510 - wine_quality_loss: 2.1728 - wine_type_loss: 0.3801 - wine_quality_root_mean_squared_error: 1.4733 - wine_type_accuracy: 0.8900 - val_loss: 2.2702 - val_wine_quality_loss: 1.9667 - val_wine_type_loss: 0.3303 - val_wine_quality_root_mean_squared_error: 1.3928 - val_wine_type_accuracy: 0.9278\n",
      "Epoch 5/180\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 2.1410 - wine_quality_loss: 1.8567 - wine_type_loss: 0.2829 - wine_quality_root_mean_squared_error: 1.3630 - wine_type_accuracy: 0.9423 - val_loss: 1.9544 - val_wine_quality_loss: 1.7243 - val_wine_type_loss: 0.2469 - val_wine_quality_root_mean_squared_error: 1.3067 - val_wine_type_accuracy: 0.9569\n",
      "Epoch 6/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 1.8720 - wine_quality_loss: 1.6656 - wine_type_loss: 0.2108 - wine_quality_root_mean_squared_error: 1.2888 - wine_type_accuracy: 0.9718 - val_loss: 1.7406 - val_wine_quality_loss: 1.5662 - val_wine_type_loss: 0.1854 - val_wine_quality_root_mean_squared_error: 1.2470 - val_wine_type_accuracy: 0.9772\n",
      "Epoch 7/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 1.6826 - wine_quality_loss: 1.5214 - wine_type_loss: 0.1604 - wine_quality_root_mean_squared_error: 1.2339 - wine_type_accuracy: 0.9819 - val_loss: 1.5808 - val_wine_quality_loss: 1.4453 - val_wine_type_loss: 0.1431 - val_wine_quality_root_mean_squared_error: 1.1990 - val_wine_type_accuracy: 0.9861\n",
      "Epoch 8/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 1.5428 - wine_quality_loss: 1.4164 - wine_type_loss: 0.1260 - wine_quality_root_mean_squared_error: 1.1902 - wine_type_accuracy: 0.9880 - val_loss: 1.4464 - val_wine_quality_loss: 1.3381 - val_wine_type_loss: 0.1137 - val_wine_quality_root_mean_squared_error: 1.1543 - val_wine_type_accuracy: 0.9873\n",
      "Epoch 9/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.4194 - wine_quality_loss: 1.3235 - wine_type_loss: 0.1030 - wine_quality_root_mean_squared_error: 1.1476 - wine_type_accuracy: 0.9883 - val_loss: 1.3289 - val_wine_quality_loss: 1.2390 - val_wine_type_loss: 0.0941 - val_wine_quality_root_mean_squared_error: 1.1111 - val_wine_type_accuracy: 0.9873\n",
      "Epoch 10/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 1.3211 - wine_quality_loss: 1.2341 - wine_type_loss: 0.0861 - wine_quality_root_mean_squared_error: 1.1113 - wine_type_accuracy: 0.9899 - val_loss: 1.2390 - val_wine_quality_loss: 1.1611 - val_wine_type_loss: 0.0796 - val_wine_quality_root_mean_squared_error: 1.0767 - val_wine_type_accuracy: 0.9873\n",
      "Epoch 11/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 1.2311 - wine_quality_loss: 1.1549 - wine_type_loss: 0.0742 - wine_quality_root_mean_squared_error: 1.0755 - wine_type_accuracy: 0.9905 - val_loss: 1.1648 - val_wine_quality_loss: 1.0964 - val_wine_type_loss: 0.0693 - val_wine_quality_root_mean_squared_error: 1.0465 - val_wine_type_accuracy: 0.9873\n",
      "Epoch 12/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.1529 - wine_quality_loss: 1.0875 - wine_type_loss: 0.0657 - wine_quality_root_mean_squared_error: 1.0426 - wine_type_accuracy: 0.9911 - val_loss: 1.0964 - val_wine_quality_loss: 1.0348 - val_wine_type_loss: 0.0615 - val_wine_quality_root_mean_squared_error: 1.0172 - val_wine_type_accuracy: 0.9899\n",
      "Epoch 13/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 1.0842 - wine_quality_loss: 1.0243 - wine_type_loss: 0.0593 - wine_quality_root_mean_squared_error: 1.0123 - wine_type_accuracy: 0.9911 - val_loss: 1.0281 - val_wine_quality_loss: 0.9716 - val_wine_type_loss: 0.0559 - val_wine_quality_root_mean_squared_error: 0.9858 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 14/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.0253 - wine_quality_loss: 0.9702 - wine_type_loss: 0.0546 - wine_quality_root_mean_squared_error: 0.9852 - wine_type_accuracy: 0.9908 - val_loss: 0.9665 - val_wine_quality_loss: 0.9139 - val_wine_type_loss: 0.0516 - val_wine_quality_root_mean_squared_error: 0.9564 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 15/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.9689 - wine_quality_loss: 0.9181 - wine_type_loss: 0.0511 - wine_quality_root_mean_squared_error: 0.9581 - wine_type_accuracy: 0.9914 - val_loss: 0.9071 - val_wine_quality_loss: 0.8578 - val_wine_type_loss: 0.0484 - val_wine_quality_root_mean_squared_error: 0.9265 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 16/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.9191 - wine_quality_loss: 0.8713 - wine_type_loss: 0.0483 - wine_quality_root_mean_squared_error: 0.9332 - wine_type_accuracy: 0.9918 - val_loss: 0.8505 - val_wine_quality_loss: 0.8041 - val_wine_type_loss: 0.0457 - val_wine_quality_root_mean_squared_error: 0.8969 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 17/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.8719 - wine_quality_loss: 0.8259 - wine_type_loss: 0.0457 - wine_quality_root_mean_squared_error: 0.9089 - wine_type_accuracy: 0.9921 - val_loss: 0.8183 - val_wine_quality_loss: 0.7732 - val_wine_type_loss: 0.0436 - val_wine_quality_root_mean_squared_error: 0.8800 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 18/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.8268 - wine_quality_loss: 0.7822 - wine_type_loss: 0.0437 - wine_quality_root_mean_squared_error: 0.8848 - wine_type_accuracy: 0.9921 - val_loss: 0.7727 - val_wine_quality_loss: 0.7296 - val_wine_type_loss: 0.0417 - val_wine_quality_root_mean_squared_error: 0.8548 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 19/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.7832 - wine_quality_loss: 0.7428 - wine_type_loss: 0.0420 - wine_quality_root_mean_squared_error: 0.8609 - wine_type_accuracy: 0.9921 - val_loss: 0.7301 - val_wine_quality_loss: 0.6883 - val_wine_type_loss: 0.0402 - val_wine_quality_root_mean_squared_error: 0.8303 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 20/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.7477 - wine_quality_loss: 0.7058 - wine_type_loss: 0.0405 - wine_quality_root_mean_squared_error: 0.8409 - wine_type_accuracy: 0.9924 - val_loss: 0.6967 - val_wine_quality_loss: 0.6562 - val_wine_type_loss: 0.0391 - val_wine_quality_root_mean_squared_error: 0.8107 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 21/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.7103 - wine_quality_loss: 0.6706 - wine_type_loss: 0.0394 - wine_quality_root_mean_squared_error: 0.8190 - wine_type_accuracy: 0.9930 - val_loss: 0.6625 - val_wine_quality_loss: 0.6231 - val_wine_type_loss: 0.0380 - val_wine_quality_root_mean_squared_error: 0.7900 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 22/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.6772 - wine_quality_loss: 0.6395 - wine_type_loss: 0.0384 - wine_quality_root_mean_squared_error: 0.7992 - wine_type_accuracy: 0.9927 - val_loss: 0.6470 - val_wine_quality_loss: 0.6080 - val_wine_type_loss: 0.0371 - val_wine_quality_root_mean_squared_error: 0.7808 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 23/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.6486 - wine_quality_loss: 0.6142 - wine_type_loss: 0.0375 - wine_quality_root_mean_squared_error: 0.7817 - wine_type_accuracy: 0.9933 - val_loss: 0.6097 - val_wine_quality_loss: 0.5718 - val_wine_type_loss: 0.0363 - val_wine_quality_root_mean_squared_error: 0.7569 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 24/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.6199 - wine_quality_loss: 0.5828 - wine_type_loss: 0.0366 - wine_quality_root_mean_squared_error: 0.7636 - wine_type_accuracy: 0.9933 - val_loss: 0.5849 - val_wine_quality_loss: 0.5474 - val_wine_type_loss: 0.0357 - val_wine_quality_root_mean_squared_error: 0.7408 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 25/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.5939 - wine_quality_loss: 0.5578 - wine_type_loss: 0.0363 - wine_quality_root_mean_squared_error: 0.7469 - wine_type_accuracy: 0.9937 - val_loss: 0.5548 - val_wine_quality_loss: 0.5184 - val_wine_type_loss: 0.0351 - val_wine_quality_root_mean_squared_error: 0.7207 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 26/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.5707 - wine_quality_loss: 0.5353 - wine_type_loss: 0.0354 - wine_quality_root_mean_squared_error: 0.7316 - wine_type_accuracy: 0.9937 - val_loss: 0.5331 - val_wine_quality_loss: 0.4970 - val_wine_type_loss: 0.0346 - val_wine_quality_root_mean_squared_error: 0.7058 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 27/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.5501 - wine_quality_loss: 0.5152 - wine_type_loss: 0.0349 - wine_quality_root_mean_squared_error: 0.7177 - wine_type_accuracy: 0.9937 - val_loss: 0.5204 - val_wine_quality_loss: 0.4843 - val_wine_type_loss: 0.0343 - val_wine_quality_root_mean_squared_error: 0.6969 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 28/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.5319 - wine_quality_loss: 0.4987 - wine_type_loss: 0.0344 - wine_quality_root_mean_squared_error: 0.7053 - wine_type_accuracy: 0.9937 - val_loss: 0.5003 - val_wine_quality_loss: 0.4646 - val_wine_type_loss: 0.0339 - val_wine_quality_root_mean_squared_error: 0.6826 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 29/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.5100 - wine_quality_loss: 0.4758 - wine_type_loss: 0.0339 - wine_quality_root_mean_squared_error: 0.6898 - wine_type_accuracy: 0.9937 - val_loss: 0.4823 - val_wine_quality_loss: 0.4471 - val_wine_type_loss: 0.0335 - val_wine_quality_root_mean_squared_error: 0.6696 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 30/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.4935 - wine_quality_loss: 0.4589 - wine_type_loss: 0.0335 - wine_quality_root_mean_squared_error: 0.6782 - wine_type_accuracy: 0.9940 - val_loss: 0.4772 - val_wine_quality_loss: 0.4424 - val_wine_type_loss: 0.0332 - val_wine_quality_root_mean_squared_error: 0.6661 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 31/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.4788 - wine_quality_loss: 0.4465 - wine_type_loss: 0.0335 - wine_quality_root_mean_squared_error: 0.6675 - wine_type_accuracy: 0.9940 - val_loss: 0.4587 - val_wine_quality_loss: 0.4241 - val_wine_type_loss: 0.0329 - val_wine_quality_root_mean_squared_error: 0.6522 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 32/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4645 - wine_quality_loss: 0.4319 - wine_type_loss: 0.0327 - wine_quality_root_mean_squared_error: 0.6571 - wine_type_accuracy: 0.9940 - val_loss: 0.4408 - val_wine_quality_loss: 0.4069 - val_wine_type_loss: 0.0326 - val_wine_quality_root_mean_squared_error: 0.6386 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 33/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4511 - wine_quality_loss: 0.4182 - wine_type_loss: 0.0324 - wine_quality_root_mean_squared_error: 0.6470 - wine_type_accuracy: 0.9940 - val_loss: 0.4326 - val_wine_quality_loss: 0.3988 - val_wine_type_loss: 0.0324 - val_wine_quality_root_mean_squared_error: 0.6324 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 34/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.4388 - wine_quality_loss: 0.4061 - wine_type_loss: 0.0320 - wine_quality_root_mean_squared_error: 0.6377 - wine_type_accuracy: 0.9940 - val_loss: 0.4235 - val_wine_quality_loss: 0.3899 - val_wine_type_loss: 0.0322 - val_wine_quality_root_mean_squared_error: 0.6253 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 35/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4296 - wine_quality_loss: 0.3985 - wine_type_loss: 0.0318 - wine_quality_root_mean_squared_error: 0.6307 - wine_type_accuracy: 0.9940 - val_loss: 0.4133 - val_wine_quality_loss: 0.3799 - val_wine_type_loss: 0.0320 - val_wine_quality_root_mean_squared_error: 0.6171 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 36/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.4189 - wine_quality_loss: 0.3869 - wine_type_loss: 0.0315 - wine_quality_root_mean_squared_error: 0.6224 - wine_type_accuracy: 0.9940 - val_loss: 0.4083 - val_wine_quality_loss: 0.3751 - val_wine_type_loss: 0.0318 - val_wine_quality_root_mean_squared_error: 0.6132 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 37/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.4096 - wine_quality_loss: 0.3782 - wine_type_loss: 0.0312 - wine_quality_root_mean_squared_error: 0.6151 - wine_type_accuracy: 0.9940 - val_loss: 0.3974 - val_wine_quality_loss: 0.3645 - val_wine_type_loss: 0.0317 - val_wine_quality_root_mean_squared_error: 0.6044 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 38/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.4029 - wine_quality_loss: 0.3724 - wine_type_loss: 0.0311 - wine_quality_root_mean_squared_error: 0.6098 - wine_type_accuracy: 0.9943 - val_loss: 0.3935 - val_wine_quality_loss: 0.3609 - val_wine_type_loss: 0.0315 - val_wine_quality_root_mean_squared_error: 0.6014 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 39/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3955 - wine_quality_loss: 0.3644 - wine_type_loss: 0.0307 - wine_quality_root_mean_squared_error: 0.6039 - wine_type_accuracy: 0.9943 - val_loss: 0.3849 - val_wine_quality_loss: 0.3525 - val_wine_type_loss: 0.0313 - val_wine_quality_root_mean_squared_error: 0.5943 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 40/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3891 - wine_quality_loss: 0.3588 - wine_type_loss: 0.0304 - wine_quality_root_mean_squared_error: 0.5989 - wine_type_accuracy: 0.9943 - val_loss: 0.3819 - val_wine_quality_loss: 0.3492 - val_wine_type_loss: 0.0312 - val_wine_quality_root_mean_squared_error: 0.5918 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 41/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3840 - wine_quality_loss: 0.3548 - wine_type_loss: 0.0302 - wine_quality_root_mean_squared_error: 0.5948 - wine_type_accuracy: 0.9943 - val_loss: 0.3802 - val_wine_quality_loss: 0.3479 - val_wine_type_loss: 0.0310 - val_wine_quality_root_mean_squared_error: 0.5905 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 42/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3776 - wine_quality_loss: 0.3472 - wine_type_loss: 0.0300 - wine_quality_root_mean_squared_error: 0.5895 - wine_type_accuracy: 0.9943 - val_loss: 0.3737 - val_wine_quality_loss: 0.3417 - val_wine_type_loss: 0.0309 - val_wine_quality_root_mean_squared_error: 0.5852 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 43/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3741 - wine_quality_loss: 0.3442 - wine_type_loss: 0.0298 - wine_quality_root_mean_squared_error: 0.5867 - wine_type_accuracy: 0.9943 - val_loss: 0.3717 - val_wine_quality_loss: 0.3399 - val_wine_type_loss: 0.0307 - val_wine_quality_root_mean_squared_error: 0.5836 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 44/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3693 - wine_quality_loss: 0.3393 - wine_type_loss: 0.0298 - wine_quality_root_mean_squared_error: 0.5829 - wine_type_accuracy: 0.9949 - val_loss: 0.3627 - val_wine_quality_loss: 0.3309 - val_wine_type_loss: 0.0307 - val_wine_quality_root_mean_squared_error: 0.5759 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 45/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3654 - wine_quality_loss: 0.3352 - wine_type_loss: 0.0294 - wine_quality_root_mean_squared_error: 0.5796 - wine_type_accuracy: 0.9943 - val_loss: 0.3669 - val_wine_quality_loss: 0.3354 - val_wine_type_loss: 0.0306 - val_wine_quality_root_mean_squared_error: 0.5797 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 46/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3619 - wine_quality_loss: 0.3331 - wine_type_loss: 0.0294 - wine_quality_root_mean_squared_error: 0.5767 - wine_type_accuracy: 0.9946 - val_loss: 0.3658 - val_wine_quality_loss: 0.3344 - val_wine_type_loss: 0.0304 - val_wine_quality_root_mean_squared_error: 0.5788 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 47/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3585 - wine_quality_loss: 0.3295 - wine_type_loss: 0.0289 - wine_quality_root_mean_squared_error: 0.5740 - wine_type_accuracy: 0.9949 - val_loss: 0.3662 - val_wine_quality_loss: 0.3347 - val_wine_type_loss: 0.0304 - val_wine_quality_root_mean_squared_error: 0.5792 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 48/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3560 - wine_quality_loss: 0.3280 - wine_type_loss: 0.0288 - wine_quality_root_mean_squared_error: 0.5719 - wine_type_accuracy: 0.9949 - val_loss: 0.3578 - val_wine_quality_loss: 0.3268 - val_wine_type_loss: 0.0302 - val_wine_quality_root_mean_squared_error: 0.5720 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 49/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3540 - wine_quality_loss: 0.3252 - wine_type_loss: 0.0286 - wine_quality_root_mean_squared_error: 0.5704 - wine_type_accuracy: 0.9946 - val_loss: 0.3544 - val_wine_quality_loss: 0.3234 - val_wine_type_loss: 0.0301 - val_wine_quality_root_mean_squared_error: 0.5691 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 50/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3512 - wine_quality_loss: 0.3225 - wine_type_loss: 0.0283 - wine_quality_root_mean_squared_error: 0.5681 - wine_type_accuracy: 0.9949 - val_loss: 0.3609 - val_wine_quality_loss: 0.3299 - val_wine_type_loss: 0.0301 - val_wine_quality_root_mean_squared_error: 0.5748 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 51/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3494 - wine_quality_loss: 0.3209 - wine_type_loss: 0.0282 - wine_quality_root_mean_squared_error: 0.5666 - wine_type_accuracy: 0.9946 - val_loss: 0.3531 - val_wine_quality_loss: 0.3223 - val_wine_type_loss: 0.0299 - val_wine_quality_root_mean_squared_error: 0.5681 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 52/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3472 - wine_quality_loss: 0.3190 - wine_type_loss: 0.0280 - wine_quality_root_mean_squared_error: 0.5649 - wine_type_accuracy: 0.9949 - val_loss: 0.3540 - val_wine_quality_loss: 0.3232 - val_wine_type_loss: 0.0299 - val_wine_quality_root_mean_squared_error: 0.5690 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 53/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3454 - wine_quality_loss: 0.3174 - wine_type_loss: 0.0279 - wine_quality_root_mean_squared_error: 0.5634 - wine_type_accuracy: 0.9949 - val_loss: 0.3538 - val_wine_quality_loss: 0.3233 - val_wine_type_loss: 0.0298 - val_wine_quality_root_mean_squared_error: 0.5689 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 54/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3433 - wine_quality_loss: 0.3153 - wine_type_loss: 0.0277 - wine_quality_root_mean_squared_error: 0.5617 - wine_type_accuracy: 0.9949 - val_loss: 0.3491 - val_wine_quality_loss: 0.3185 - val_wine_type_loss: 0.0297 - val_wine_quality_root_mean_squared_error: 0.5648 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 55/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3412 - wine_quality_loss: 0.3131 - wine_type_loss: 0.0275 - wine_quality_root_mean_squared_error: 0.5600 - wine_type_accuracy: 0.9949 - val_loss: 0.3559 - val_wine_quality_loss: 0.3252 - val_wine_type_loss: 0.0296 - val_wine_quality_root_mean_squared_error: 0.5708 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 56/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3406 - wine_quality_loss: 0.3134 - wine_type_loss: 0.0274 - wine_quality_root_mean_squared_error: 0.5595 - wine_type_accuracy: 0.9949 - val_loss: 0.3462 - val_wine_quality_loss: 0.3159 - val_wine_type_loss: 0.0296 - val_wine_quality_root_mean_squared_error: 0.5624 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 57/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3376 - wine_quality_loss: 0.3097 - wine_type_loss: 0.0274 - wine_quality_root_mean_squared_error: 0.5571 - wine_type_accuracy: 0.9949 - val_loss: 0.3583 - val_wine_quality_loss: 0.3282 - val_wine_type_loss: 0.0295 - val_wine_quality_root_mean_squared_error: 0.5731 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 58/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3376 - wine_quality_loss: 0.3108 - wine_type_loss: 0.0270 - wine_quality_root_mean_squared_error: 0.5572 - wine_type_accuracy: 0.9949 - val_loss: 0.3463 - val_wine_quality_loss: 0.3159 - val_wine_type_loss: 0.0293 - val_wine_quality_root_mean_squared_error: 0.5626 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 59/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3348 - wine_quality_loss: 0.3075 - wine_type_loss: 0.0269 - wine_quality_root_mean_squared_error: 0.5548 - wine_type_accuracy: 0.9949 - val_loss: 0.3517 - val_wine_quality_loss: 0.3219 - val_wine_type_loss: 0.0293 - val_wine_quality_root_mean_squared_error: 0.5675 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 60/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3360 - wine_quality_loss: 0.3092 - wine_type_loss: 0.0267 - wine_quality_root_mean_squared_error: 0.5560 - wine_type_accuracy: 0.9949 - val_loss: 0.3430 - val_wine_quality_loss: 0.3132 - val_wine_type_loss: 0.0293 - val_wine_quality_root_mean_squared_error: 0.5598 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 61/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3344 - wine_quality_loss: 0.3073 - wine_type_loss: 0.0266 - wine_quality_root_mean_squared_error: 0.5547 - wine_type_accuracy: 0.9949 - val_loss: 0.3524 - val_wine_quality_loss: 0.3224 - val_wine_type_loss: 0.0291 - val_wine_quality_root_mean_squared_error: 0.5682 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 62/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3327 - wine_quality_loss: 0.3061 - wine_type_loss: 0.0264 - wine_quality_root_mean_squared_error: 0.5534 - wine_type_accuracy: 0.9949 - val_loss: 0.3459 - val_wine_quality_loss: 0.3160 - val_wine_type_loss: 0.0292 - val_wine_quality_root_mean_squared_error: 0.5625 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 63/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3314 - wine_quality_loss: 0.3048 - wine_type_loss: 0.0264 - wine_quality_root_mean_squared_error: 0.5522 - wine_type_accuracy: 0.9949 - val_loss: 0.3491 - val_wine_quality_loss: 0.3195 - val_wine_type_loss: 0.0291 - val_wine_quality_root_mean_squared_error: 0.5654 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 64/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3299 - wine_quality_loss: 0.3034 - wine_type_loss: 0.0261 - wine_quality_root_mean_squared_error: 0.5511 - wine_type_accuracy: 0.9949 - val_loss: 0.3483 - val_wine_quality_loss: 0.3189 - val_wine_type_loss: 0.0290 - val_wine_quality_root_mean_squared_error: 0.5648 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 65/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3292 - wine_quality_loss: 0.3038 - wine_type_loss: 0.0259 - wine_quality_root_mean_squared_error: 0.5507 - wine_type_accuracy: 0.9949 - val_loss: 0.3456 - val_wine_quality_loss: 0.3157 - val_wine_type_loss: 0.0289 - val_wine_quality_root_mean_squared_error: 0.5624 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 66/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.3282 - wine_quality_loss: 0.3024 - wine_type_loss: 0.0259 - wine_quality_root_mean_squared_error: 0.5498 - wine_type_accuracy: 0.9949 - val_loss: 0.3451 - val_wine_quality_loss: 0.3155 - val_wine_type_loss: 0.0290 - val_wine_quality_root_mean_squared_error: 0.5620 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 67/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3276 - wine_quality_loss: 0.3023 - wine_type_loss: 0.0257 - wine_quality_root_mean_squared_error: 0.5494 - wine_type_accuracy: 0.9949 - val_loss: 0.3422 - val_wine_quality_loss: 0.3129 - val_wine_type_loss: 0.0288 - val_wine_quality_root_mean_squared_error: 0.5595 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 68/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3266 - wine_quality_loss: 0.3010 - wine_type_loss: 0.0255 - wine_quality_root_mean_squared_error: 0.5486 - wine_type_accuracy: 0.9949 - val_loss: 0.3534 - val_wine_quality_loss: 0.3243 - val_wine_type_loss: 0.0288 - val_wine_quality_root_mean_squared_error: 0.5695 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 69/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3248 - wine_quality_loss: 0.2996 - wine_type_loss: 0.0254 - wine_quality_root_mean_squared_error: 0.5471 - wine_type_accuracy: 0.9949 - val_loss: 0.3388 - val_wine_quality_loss: 0.3093 - val_wine_type_loss: 0.0288 - val_wine_quality_root_mean_squared_error: 0.5564 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 70/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3235 - wine_quality_loss: 0.2991 - wine_type_loss: 0.0253 - wine_quality_root_mean_squared_error: 0.5460 - wine_type_accuracy: 0.9949 - val_loss: 0.3388 - val_wine_quality_loss: 0.3095 - val_wine_type_loss: 0.0287 - val_wine_quality_root_mean_squared_error: 0.5565 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 71/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3229 - wine_quality_loss: 0.2975 - wine_type_loss: 0.0252 - wine_quality_root_mean_squared_error: 0.5456 - wine_type_accuracy: 0.9949 - val_loss: 0.3426 - val_wine_quality_loss: 0.3132 - val_wine_type_loss: 0.0287 - val_wine_quality_root_mean_squared_error: 0.5600 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 72/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3230 - wine_quality_loss: 0.2984 - wine_type_loss: 0.0249 - wine_quality_root_mean_squared_error: 0.5459 - wine_type_accuracy: 0.9952 - val_loss: 0.3384 - val_wine_quality_loss: 0.3094 - val_wine_type_loss: 0.0286 - val_wine_quality_root_mean_squared_error: 0.5562 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 73/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3209 - wine_quality_loss: 0.2959 - wine_type_loss: 0.0249 - wine_quality_root_mean_squared_error: 0.5440 - wine_type_accuracy: 0.9949 - val_loss: 0.3419 - val_wine_quality_loss: 0.3124 - val_wine_type_loss: 0.0286 - val_wine_quality_root_mean_squared_error: 0.5594 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 74/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3204 - wine_quality_loss: 0.2957 - wine_type_loss: 0.0247 - wine_quality_root_mean_squared_error: 0.5436 - wine_type_accuracy: 0.9952 - val_loss: 0.3424 - val_wine_quality_loss: 0.3133 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5599 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 75/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3207 - wine_quality_loss: 0.2965 - wine_type_loss: 0.0246 - wine_quality_root_mean_squared_error: 0.5441 - wine_type_accuracy: 0.9949 - val_loss: 0.3488 - val_wine_quality_loss: 0.3195 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5656 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 76/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3195 - wine_quality_loss: 0.2952 - wine_type_loss: 0.0245 - wine_quality_root_mean_squared_error: 0.5432 - wine_type_accuracy: 0.9952 - val_loss: 0.3393 - val_wine_quality_loss: 0.3103 - val_wine_type_loss: 0.0284 - val_wine_quality_root_mean_squared_error: 0.5572 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 77/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3191 - wine_quality_loss: 0.2946 - wine_type_loss: 0.0243 - wine_quality_root_mean_squared_error: 0.5429 - wine_type_accuracy: 0.9956 - val_loss: 0.3383 - val_wine_quality_loss: 0.3091 - val_wine_type_loss: 0.0285 - val_wine_quality_root_mean_squared_error: 0.5563 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 78/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3189 - wine_quality_loss: 0.2944 - wine_type_loss: 0.0242 - wine_quality_root_mean_squared_error: 0.5428 - wine_type_accuracy: 0.9952 - val_loss: 0.3363 - val_wine_quality_loss: 0.3074 - val_wine_type_loss: 0.0284 - val_wine_quality_root_mean_squared_error: 0.5546 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 79/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3172 - wine_quality_loss: 0.2934 - wine_type_loss: 0.0241 - wine_quality_root_mean_squared_error: 0.5414 - wine_type_accuracy: 0.9956 - val_loss: 0.3395 - val_wine_quality_loss: 0.3104 - val_wine_type_loss: 0.0284 - val_wine_quality_root_mean_squared_error: 0.5575 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 80/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3170 - wine_quality_loss: 0.2933 - wine_type_loss: 0.0241 - wine_quality_root_mean_squared_error: 0.5413 - wine_type_accuracy: 0.9952 - val_loss: 0.3429 - val_wine_quality_loss: 0.3142 - val_wine_type_loss: 0.0282 - val_wine_quality_root_mean_squared_error: 0.5607 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 81/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3156 - wine_quality_loss: 0.2911 - wine_type_loss: 0.0238 - wine_quality_root_mean_squared_error: 0.5400 - wine_type_accuracy: 0.9956 - val_loss: 0.3350 - val_wine_quality_loss: 0.3064 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5536 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 82/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3134 - wine_quality_loss: 0.2895 - wine_type_loss: 0.0237 - wine_quality_root_mean_squared_error: 0.5382 - wine_type_accuracy: 0.9956 - val_loss: 0.3326 - val_wine_quality_loss: 0.3040 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5515 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 83/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3140 - wine_quality_loss: 0.2901 - wine_type_loss: 0.0236 - wine_quality_root_mean_squared_error: 0.5387 - wine_type_accuracy: 0.9952 - val_loss: 0.3360 - val_wine_quality_loss: 0.3074 - val_wine_type_loss: 0.0281 - val_wine_quality_root_mean_squared_error: 0.5546 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 84/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3122 - wine_quality_loss: 0.2878 - wine_type_loss: 0.0235 - wine_quality_root_mean_squared_error: 0.5372 - wine_type_accuracy: 0.9959 - val_loss: 0.3544 - val_wine_quality_loss: 0.3256 - val_wine_type_loss: 0.0280 - val_wine_quality_root_mean_squared_error: 0.5710 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 85/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3129 - wine_quality_loss: 0.2892 - wine_type_loss: 0.0233 - wine_quality_root_mean_squared_error: 0.5381 - wine_type_accuracy: 0.9959 - val_loss: 0.3408 - val_wine_quality_loss: 0.3123 - val_wine_type_loss: 0.0280 - val_wine_quality_root_mean_squared_error: 0.5590 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 86/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3119 - wine_quality_loss: 0.2885 - wine_type_loss: 0.0232 - wine_quality_root_mean_squared_error: 0.5372 - wine_type_accuracy: 0.9959 - val_loss: 0.3466 - val_wine_quality_loss: 0.3180 - val_wine_type_loss: 0.0279 - val_wine_quality_root_mean_squared_error: 0.5642 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 87/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3108 - wine_quality_loss: 0.2872 - wine_type_loss: 0.0231 - wine_quality_root_mean_squared_error: 0.5363 - wine_type_accuracy: 0.9962 - val_loss: 0.3464 - val_wine_quality_loss: 0.3182 - val_wine_type_loss: 0.0279 - val_wine_quality_root_mean_squared_error: 0.5641 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 88/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3109 - wine_quality_loss: 0.2881 - wine_type_loss: 0.0230 - wine_quality_root_mean_squared_error: 0.5365 - wine_type_accuracy: 0.9962 - val_loss: 0.3337 - val_wine_quality_loss: 0.3054 - val_wine_type_loss: 0.0278 - val_wine_quality_root_mean_squared_error: 0.5527 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 89/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3103 - wine_quality_loss: 0.2872 - wine_type_loss: 0.0228 - wine_quality_root_mean_squared_error: 0.5361 - wine_type_accuracy: 0.9959 - val_loss: 0.3328 - val_wine_quality_loss: 0.3046 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5520 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 90/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3089 - wine_quality_loss: 0.2864 - wine_type_loss: 0.0228 - wine_quality_root_mean_squared_error: 0.5348 - wine_type_accuracy: 0.9962 - val_loss: 0.3376 - val_wine_quality_loss: 0.3096 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5564 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 91/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3096 - wine_quality_loss: 0.2864 - wine_type_loss: 0.0226 - wine_quality_root_mean_squared_error: 0.5356 - wine_type_accuracy: 0.9962 - val_loss: 0.3375 - val_wine_quality_loss: 0.3092 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5563 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 92/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.3086 - wine_quality_loss: 0.2860 - wine_type_loss: 0.0225 - wine_quality_root_mean_squared_error: 0.5347 - wine_type_accuracy: 0.9962 - val_loss: 0.3370 - val_wine_quality_loss: 0.3088 - val_wine_type_loss: 0.0277 - val_wine_quality_root_mean_squared_error: 0.5559 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 93/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3071 - wine_quality_loss: 0.2852 - wine_type_loss: 0.0224 - wine_quality_root_mean_squared_error: 0.5335 - wine_type_accuracy: 0.9962 - val_loss: 0.3321 - val_wine_quality_loss: 0.3040 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5515 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 94/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3062 - wine_quality_loss: 0.2841 - wine_type_loss: 0.0223 - wine_quality_root_mean_squared_error: 0.5328 - wine_type_accuracy: 0.9962 - val_loss: 0.3414 - val_wine_quality_loss: 0.3132 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5598 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 95/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3059 - wine_quality_loss: 0.2835 - wine_type_loss: 0.0222 - wine_quality_root_mean_squared_error: 0.5326 - wine_type_accuracy: 0.9962 - val_loss: 0.3466 - val_wine_quality_loss: 0.3183 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5645 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 96/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.3053 - wine_quality_loss: 0.2834 - wine_type_loss: 0.0221 - wine_quality_root_mean_squared_error: 0.5321 - wine_type_accuracy: 0.9962 - val_loss: 0.3352 - val_wine_quality_loss: 0.3071 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5543 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 97/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.3041 - wine_quality_loss: 0.2819 - wine_type_loss: 0.0220 - wine_quality_root_mean_squared_error: 0.5311 - wine_type_accuracy: 0.9962 - val_loss: 0.3343 - val_wine_quality_loss: 0.3064 - val_wine_type_loss: 0.0276 - val_wine_quality_root_mean_squared_error: 0.5535 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 98/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3052 - wine_quality_loss: 0.2830 - wine_type_loss: 0.0219 - wine_quality_root_mean_squared_error: 0.5323 - wine_type_accuracy: 0.9962 - val_loss: 0.3320 - val_wine_quality_loss: 0.3043 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5515 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 99/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3048 - wine_quality_loss: 0.2830 - wine_type_loss: 0.0217 - wine_quality_root_mean_squared_error: 0.5320 - wine_type_accuracy: 0.9962 - val_loss: 0.3315 - val_wine_quality_loss: 0.3035 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5510 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 100/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3036 - wine_quality_loss: 0.2816 - wine_type_loss: 0.0218 - wine_quality_root_mean_squared_error: 0.5310 - wine_type_accuracy: 0.9962 - val_loss: 0.3383 - val_wine_quality_loss: 0.3101 - val_wine_type_loss: 0.0275 - val_wine_quality_root_mean_squared_error: 0.5571 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 101/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.3026 - wine_quality_loss: 0.2806 - wine_type_loss: 0.0216 - wine_quality_root_mean_squared_error: 0.5300 - wine_type_accuracy: 0.9962 - val_loss: 0.3340 - val_wine_quality_loss: 0.3061 - val_wine_type_loss: 0.0274 - val_wine_quality_root_mean_squared_error: 0.5534 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 102/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3021 - wine_quality_loss: 0.2805 - wine_type_loss: 0.0226 - wine_quality_root_mean_squared_error: 0.5297 - wine_type_accuracy: 0.9962 - val_loss: 0.3454 - val_wine_quality_loss: 0.3177 - val_wine_type_loss: 0.0272 - val_wine_quality_root_mean_squared_error: 0.5638 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 103/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3014 - wine_quality_loss: 0.2797 - wine_type_loss: 0.0213 - wine_quality_root_mean_squared_error: 0.5292 - wine_type_accuracy: 0.9962 - val_loss: 0.3293 - val_wine_quality_loss: 0.3016 - val_wine_type_loss: 0.0273 - val_wine_quality_root_mean_squared_error: 0.5492 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 104/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3005 - wine_quality_loss: 0.2800 - wine_type_loss: 0.0212 - wine_quality_root_mean_squared_error: 0.5284 - wine_type_accuracy: 0.9962 - val_loss: 0.3319 - val_wine_quality_loss: 0.3042 - val_wine_type_loss: 0.0273 - val_wine_quality_root_mean_squared_error: 0.5516 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 105/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.3012 - wine_quality_loss: 0.2805 - wine_type_loss: 0.0211 - wine_quality_root_mean_squared_error: 0.5291 - wine_type_accuracy: 0.9962 - val_loss: 0.3360 - val_wine_quality_loss: 0.3082 - val_wine_type_loss: 0.0271 - val_wine_quality_root_mean_squared_error: 0.5554 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 106/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.3005 - wine_quality_loss: 0.2793 - wine_type_loss: 0.0210 - wine_quality_root_mean_squared_error: 0.5286 - wine_type_accuracy: 0.9962 - val_loss: 0.3287 - val_wine_quality_loss: 0.3012 - val_wine_type_loss: 0.0271 - val_wine_quality_root_mean_squared_error: 0.5489 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 107/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2996 - wine_quality_loss: 0.2785 - wine_type_loss: 0.0209 - wine_quality_root_mean_squared_error: 0.5278 - wine_type_accuracy: 0.9962 - val_loss: 0.3290 - val_wine_quality_loss: 0.3016 - val_wine_type_loss: 0.0271 - val_wine_quality_root_mean_squared_error: 0.5492 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 108/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2992 - wine_quality_loss: 0.2780 - wine_type_loss: 0.0208 - wine_quality_root_mean_squared_error: 0.5276 - wine_type_accuracy: 0.9962 - val_loss: 0.3323 - val_wine_quality_loss: 0.3048 - val_wine_type_loss: 0.0271 - val_wine_quality_root_mean_squared_error: 0.5522 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 109/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2983 - wine_quality_loss: 0.2783 - wine_type_loss: 0.0207 - wine_quality_root_mean_squared_error: 0.5268 - wine_type_accuracy: 0.9962 - val_loss: 0.3298 - val_wine_quality_loss: 0.3024 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5499 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 110/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2966 - wine_quality_loss: 0.2755 - wine_type_loss: 0.0206 - wine_quality_root_mean_squared_error: 0.5253 - wine_type_accuracy: 0.9962 - val_loss: 0.3326 - val_wine_quality_loss: 0.3049 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5525 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 111/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2971 - wine_quality_loss: 0.2764 - wine_type_loss: 0.0205 - wine_quality_root_mean_squared_error: 0.5259 - wine_type_accuracy: 0.9962 - val_loss: 0.3305 - val_wine_quality_loss: 0.3029 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5506 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 112/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2969 - wine_quality_loss: 0.2769 - wine_type_loss: 0.0203 - wine_quality_root_mean_squared_error: 0.5258 - wine_type_accuracy: 0.9965 - val_loss: 0.3443 - val_wine_quality_loss: 0.3165 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5630 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 113/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2968 - wine_quality_loss: 0.2761 - wine_type_loss: 0.0203 - wine_quality_root_mean_squared_error: 0.5258 - wine_type_accuracy: 0.9965 - val_loss: 0.3278 - val_wine_quality_loss: 0.3005 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5482 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 114/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2953 - wine_quality_loss: 0.2752 - wine_type_loss: 0.0201 - wine_quality_root_mean_squared_error: 0.5245 - wine_type_accuracy: 0.9965 - val_loss: 0.3291 - val_wine_quality_loss: 0.3019 - val_wine_type_loss: 0.0270 - val_wine_quality_root_mean_squared_error: 0.5493 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 115/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2951 - wine_quality_loss: 0.2755 - wine_type_loss: 0.0201 - wine_quality_root_mean_squared_error: 0.5244 - wine_type_accuracy: 0.9965 - val_loss: 0.3314 - val_wine_quality_loss: 0.3040 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5515 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 116/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2958 - wine_quality_loss: 0.2759 - wine_type_loss: 0.0220 - wine_quality_root_mean_squared_error: 0.5251 - wine_type_accuracy: 0.9965 - val_loss: 0.3351 - val_wine_quality_loss: 0.3077 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5549 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 117/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2945 - wine_quality_loss: 0.2741 - wine_type_loss: 0.0199 - wine_quality_root_mean_squared_error: 0.5240 - wine_type_accuracy: 0.9965 - val_loss: 0.3302 - val_wine_quality_loss: 0.3029 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5504 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 118/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2940 - wine_quality_loss: 0.2737 - wine_type_loss: 0.0198 - wine_quality_root_mean_squared_error: 0.5235 - wine_type_accuracy: 0.9965 - val_loss: 0.3351 - val_wine_quality_loss: 0.3079 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5549 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 119/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2946 - wine_quality_loss: 0.2745 - wine_type_loss: 0.0197 - wine_quality_root_mean_squared_error: 0.5242 - wine_type_accuracy: 0.9965 - val_loss: 0.3290 - val_wine_quality_loss: 0.3016 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5494 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 120/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2933 - wine_quality_loss: 0.2735 - wine_type_loss: 0.0197 - wine_quality_root_mean_squared_error: 0.5230 - wine_type_accuracy: 0.9965 - val_loss: 0.3297 - val_wine_quality_loss: 0.3026 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5500 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 121/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2922 - wine_quality_loss: 0.2728 - wine_type_loss: 0.0196 - wine_quality_root_mean_squared_error: 0.5221 - wine_type_accuracy: 0.9965 - val_loss: 0.3302 - val_wine_quality_loss: 0.3031 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5505 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 122/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2919 - wine_quality_loss: 0.2729 - wine_type_loss: 0.0194 - wine_quality_root_mean_squared_error: 0.5220 - wine_type_accuracy: 0.9965 - val_loss: 0.3309 - val_wine_quality_loss: 0.3035 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5511 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 123/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2918 - wine_quality_loss: 0.2720 - wine_type_loss: 0.0193 - wine_quality_root_mean_squared_error: 0.5219 - wine_type_accuracy: 0.9965 - val_loss: 0.3294 - val_wine_quality_loss: 0.3020 - val_wine_type_loss: 0.0269 - val_wine_quality_root_mean_squared_error: 0.5497 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 124/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2910 - wine_quality_loss: 0.2718 - wine_type_loss: 0.0193 - wine_quality_root_mean_squared_error: 0.5212 - wine_type_accuracy: 0.9965 - val_loss: 0.3280 - val_wine_quality_loss: 0.3009 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5485 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 125/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2908 - wine_quality_loss: 0.2715 - wine_type_loss: 0.0197 - wine_quality_root_mean_squared_error: 0.5211 - wine_type_accuracy: 0.9965 - val_loss: 0.3322 - val_wine_quality_loss: 0.3049 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5523 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 126/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2911 - wine_quality_loss: 0.2722 - wine_type_loss: 0.0191 - wine_quality_root_mean_squared_error: 0.5214 - wine_type_accuracy: 0.9965 - val_loss: 0.3307 - val_wine_quality_loss: 0.3035 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5509 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 127/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2902 - wine_quality_loss: 0.2717 - wine_type_loss: 0.0191 - wine_quality_root_mean_squared_error: 0.5206 - wine_type_accuracy: 0.9965 - val_loss: 0.3305 - val_wine_quality_loss: 0.3036 - val_wine_type_loss: 0.0268 - val_wine_quality_root_mean_squared_error: 0.5508 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 128/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2894 - wine_quality_loss: 0.2703 - wine_type_loss: 0.0189 - wine_quality_root_mean_squared_error: 0.5200 - wine_type_accuracy: 0.9965 - val_loss: 0.3336 - val_wine_quality_loss: 0.3064 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5537 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 129/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2891 - wine_quality_loss: 0.2702 - wine_type_loss: 0.0189 - wine_quality_root_mean_squared_error: 0.5198 - wine_type_accuracy: 0.9965 - val_loss: 0.3297 - val_wine_quality_loss: 0.3026 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5502 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 130/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2875 - wine_quality_loss: 0.2687 - wine_type_loss: 0.0188 - wine_quality_root_mean_squared_error: 0.5183 - wine_type_accuracy: 0.9965 - val_loss: 0.3402 - val_wine_quality_loss: 0.3129 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5597 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 131/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2881 - wine_quality_loss: 0.2702 - wine_type_loss: 0.0187 - wine_quality_root_mean_squared_error: 0.5190 - wine_type_accuracy: 0.9965 - val_loss: 0.3321 - val_wine_quality_loss: 0.3051 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5524 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 132/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2870 - wine_quality_loss: 0.2682 - wine_type_loss: 0.0188 - wine_quality_root_mean_squared_error: 0.5180 - wine_type_accuracy: 0.9965 - val_loss: 0.3364 - val_wine_quality_loss: 0.3093 - val_wine_type_loss: 0.0267 - val_wine_quality_root_mean_squared_error: 0.5563 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 133/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2867 - wine_quality_loss: 0.2681 - wine_type_loss: 0.0186 - wine_quality_root_mean_squared_error: 0.5178 - wine_type_accuracy: 0.9965 - val_loss: 0.3330 - val_wine_quality_loss: 0.3061 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5532 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 134/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2870 - wine_quality_loss: 0.2685 - wine_type_loss: 0.0185 - wine_quality_root_mean_squared_error: 0.5181 - wine_type_accuracy: 0.9965 - val_loss: 0.3298 - val_wine_quality_loss: 0.3026 - val_wine_type_loss: 0.0266 - val_wine_quality_root_mean_squared_error: 0.5503 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 135/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2855 - wine_quality_loss: 0.2672 - wine_type_loss: 0.0194 - wine_quality_root_mean_squared_error: 0.5168 - wine_type_accuracy: 0.9965 - val_loss: 0.3325 - val_wine_quality_loss: 0.3056 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5528 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 136/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2862 - wine_quality_loss: 0.2676 - wine_type_loss: 0.0183 - wine_quality_root_mean_squared_error: 0.5175 - wine_type_accuracy: 0.9965 - val_loss: 0.3274 - val_wine_quality_loss: 0.3005 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5482 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 137/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2859 - wine_quality_loss: 0.2682 - wine_type_loss: 0.0182 - wine_quality_root_mean_squared_error: 0.5173 - wine_type_accuracy: 0.9965 - val_loss: 0.3300 - val_wine_quality_loss: 0.3031 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5507 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 138/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2841 - wine_quality_loss: 0.2657 - wine_type_loss: 0.0181 - wine_quality_root_mean_squared_error: 0.5158 - wine_type_accuracy: 0.9965 - val_loss: 0.3410 - val_wine_quality_loss: 0.3146 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5606 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 139/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2850 - wine_quality_loss: 0.2666 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5167 - wine_type_accuracy: 0.9965 - val_loss: 0.3269 - val_wine_quality_loss: 0.3001 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5478 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 140/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2845 - wine_quality_loss: 0.2664 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5163 - wine_type_accuracy: 0.9968 - val_loss: 0.3314 - val_wine_quality_loss: 0.3047 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5519 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 141/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2833 - wine_quality_loss: 0.2649 - wine_type_loss: 0.0179 - wine_quality_root_mean_squared_error: 0.5152 - wine_type_accuracy: 0.9965 - val_loss: 0.3283 - val_wine_quality_loss: 0.3012 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5490 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 142/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2824 - wine_quality_loss: 0.2645 - wine_type_loss: 0.0177 - wine_quality_root_mean_squared_error: 0.5144 - wine_type_accuracy: 0.9968 - val_loss: 0.3327 - val_wine_quality_loss: 0.3060 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5530 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 143/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2827 - wine_quality_loss: 0.2648 - wine_type_loss: 0.0177 - wine_quality_root_mean_squared_error: 0.5147 - wine_type_accuracy: 0.9968 - val_loss: 0.3339 - val_wine_quality_loss: 0.3070 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5541 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 144/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2830 - wine_quality_loss: 0.2659 - wine_type_loss: 0.0176 - wine_quality_root_mean_squared_error: 0.5151 - wine_type_accuracy: 0.9965 - val_loss: 0.3314 - val_wine_quality_loss: 0.3046 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5519 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 145/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2830 - wine_quality_loss: 0.2658 - wine_type_loss: 0.0175 - wine_quality_root_mean_squared_error: 0.5152 - wine_type_accuracy: 0.9965 - val_loss: 0.3306 - val_wine_quality_loss: 0.3038 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5512 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 146/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2822 - wine_quality_loss: 0.2646 - wine_type_loss: 0.0177 - wine_quality_root_mean_squared_error: 0.5145 - wine_type_accuracy: 0.9965 - val_loss: 0.3293 - val_wine_quality_loss: 0.3025 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5500 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 147/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2796 - wine_quality_loss: 0.2625 - wine_type_loss: 0.0173 - wine_quality_root_mean_squared_error: 0.5121 - wine_type_accuracy: 0.9968 - val_loss: 0.3345 - val_wine_quality_loss: 0.3075 - val_wine_type_loss: 0.0265 - val_wine_quality_root_mean_squared_error: 0.5547 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 148/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2806 - wine_quality_loss: 0.2635 - wine_type_loss: 0.0173 - wine_quality_root_mean_squared_error: 0.5131 - wine_type_accuracy: 0.9968 - val_loss: 0.3302 - val_wine_quality_loss: 0.3033 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5509 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 149/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2803 - wine_quality_loss: 0.2633 - wine_type_loss: 0.0172 - wine_quality_root_mean_squared_error: 0.5130 - wine_type_accuracy: 0.9968 - val_loss: 0.3318 - val_wine_quality_loss: 0.3051 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5524 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 150/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2796 - wine_quality_loss: 0.2625 - wine_type_loss: 0.0170 - wine_quality_root_mean_squared_error: 0.5124 - wine_type_accuracy: 0.9968 - val_loss: 0.3318 - val_wine_quality_loss: 0.3051 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5523 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 151/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2805 - wine_quality_loss: 0.2635 - wine_type_loss: 0.0171 - wine_quality_root_mean_squared_error: 0.5132 - wine_type_accuracy: 0.9968 - val_loss: 0.3263 - val_wine_quality_loss: 0.2996 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5473 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 152/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2807 - wine_quality_loss: 0.2647 - wine_type_loss: 0.0170 - wine_quality_root_mean_squared_error: 0.5135 - wine_type_accuracy: 0.9968 - val_loss: 0.3356 - val_wine_quality_loss: 0.3088 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5558 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 153/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2790 - wine_quality_loss: 0.2620 - wine_type_loss: 0.0169 - wine_quality_root_mean_squared_error: 0.5119 - wine_type_accuracy: 0.9968 - val_loss: 0.3275 - val_wine_quality_loss: 0.3008 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5485 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 154/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2780 - wine_quality_loss: 0.2609 - wine_type_loss: 0.0168 - wine_quality_root_mean_squared_error: 0.5111 - wine_type_accuracy: 0.9968 - val_loss: 0.3339 - val_wine_quality_loss: 0.3068 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5543 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 155/180\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.2776 - wine_quality_loss: 0.2604 - wine_type_loss: 0.0168 - wine_quality_root_mean_squared_error: 0.5107 - wine_type_accuracy: 0.9968 - val_loss: 0.3269 - val_wine_quality_loss: 0.3002 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5479 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 156/180\n",
      "3155/3155 [==============================] - 0s 71us/sample - loss: 0.2775 - wine_quality_loss: 0.2605 - wine_type_loss: 0.0167 - wine_quality_root_mean_squared_error: 0.5107 - wine_type_accuracy: 0.9968 - val_loss: 0.3294 - val_wine_quality_loss: 0.3027 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5502 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 157/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2770 - wine_quality_loss: 0.2597 - wine_type_loss: 0.0166 - wine_quality_root_mean_squared_error: 0.5103 - wine_type_accuracy: 0.9968 - val_loss: 0.3302 - val_wine_quality_loss: 0.3038 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5509 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 158/180\n",
      "3155/3155 [==============================] - 0s 87us/sample - loss: 0.2767 - wine_quality_loss: 0.2600 - wine_type_loss: 0.0165 - wine_quality_root_mean_squared_error: 0.5100 - wine_type_accuracy: 0.9968 - val_loss: 0.3399 - val_wine_quality_loss: 0.3134 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5597 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 159/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2771 - wine_quality_loss: 0.2602 - wine_type_loss: 0.0165 - wine_quality_root_mean_squared_error: 0.5104 - wine_type_accuracy: 0.9968 - val_loss: 0.3274 - val_wine_quality_loss: 0.3007 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5484 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 160/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2749 - wine_quality_loss: 0.2583 - wine_type_loss: 0.0164 - wine_quality_root_mean_squared_error: 0.5084 - wine_type_accuracy: 0.9968 - val_loss: 0.3278 - val_wine_quality_loss: 0.3009 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5487 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 161/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2756 - wine_quality_loss: 0.2592 - wine_type_loss: 0.0163 - wine_quality_root_mean_squared_error: 0.5092 - wine_type_accuracy: 0.9968 - val_loss: 0.3261 - val_wine_quality_loss: 0.2995 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5472 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 162/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2757 - wine_quality_loss: 0.2595 - wine_type_loss: 0.0162 - wine_quality_root_mean_squared_error: 0.5093 - wine_type_accuracy: 0.9968 - val_loss: 0.3308 - val_wine_quality_loss: 0.3039 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5514 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 163/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2740 - wine_quality_loss: 0.2583 - wine_type_loss: 0.0162 - wine_quality_root_mean_squared_error: 0.5077 - wine_type_accuracy: 0.9968 - val_loss: 0.3280 - val_wine_quality_loss: 0.3011 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5489 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 164/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.2748 - wine_quality_loss: 0.2583 - wine_type_loss: 0.0162 - wine_quality_root_mean_squared_error: 0.5085 - wine_type_accuracy: 0.9968 - val_loss: 0.3399 - val_wine_quality_loss: 0.3128 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5596 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 165/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2738 - wine_quality_loss: 0.2578 - wine_type_loss: 0.0160 - wine_quality_root_mean_squared_error: 0.5077 - wine_type_accuracy: 0.9968 - val_loss: 0.3272 - val_wine_quality_loss: 0.3004 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5481 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 166/180\n",
      "3155/3155 [==============================] - 0s 72us/sample - loss: 0.2735 - wine_quality_loss: 0.2577 - wine_type_loss: 0.0160 - wine_quality_root_mean_squared_error: 0.5074 - wine_type_accuracy: 0.9968 - val_loss: 0.3338 - val_wine_quality_loss: 0.3072 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5542 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 167/180\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.2730 - wine_quality_loss: 0.2570 - wine_type_loss: 0.0159 - wine_quality_root_mean_squared_error: 0.5070 - wine_type_accuracy: 0.9968 - val_loss: 0.3293 - val_wine_quality_loss: 0.3027 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5501 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 168/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2731 - wine_quality_loss: 0.2574 - wine_type_loss: 0.0158 - wine_quality_root_mean_squared_error: 0.5072 - wine_type_accuracy: 0.9968 - val_loss: 0.3242 - val_wine_quality_loss: 0.2975 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5454 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 169/180\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2721 - wine_quality_loss: 0.2563 - wine_type_loss: 0.0158 - wine_quality_root_mean_squared_error: 0.5062 - wine_type_accuracy: 0.9968 - val_loss: 0.3300 - val_wine_quality_loss: 0.3033 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5507 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 170/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2723 - wine_quality_loss: 0.2568 - wine_type_loss: 0.0157 - wine_quality_root_mean_squared_error: 0.5065 - wine_type_accuracy: 0.9968 - val_loss: 0.3288 - val_wine_quality_loss: 0.3020 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5496 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 171/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2723 - wine_quality_loss: 0.2564 - wine_type_loss: 0.0156 - wine_quality_root_mean_squared_error: 0.5066 - wine_type_accuracy: 0.9968 - val_loss: 0.3273 - val_wine_quality_loss: 0.3009 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5483 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 172/180\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.2720 - wine_quality_loss: 0.2561 - wine_type_loss: 0.0156 - wine_quality_root_mean_squared_error: 0.5063 - wine_type_accuracy: 0.9968 - val_loss: 0.3263 - val_wine_quality_loss: 0.2998 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5474 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 173/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2707 - wine_quality_loss: 0.2556 - wine_type_loss: 0.0155 - wine_quality_root_mean_squared_error: 0.5052 - wine_type_accuracy: 0.9968 - val_loss: 0.3333 - val_wine_quality_loss: 0.3068 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5537 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 174/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2704 - wine_quality_loss: 0.2545 - wine_type_loss: 0.0154 - wine_quality_root_mean_squared_error: 0.5049 - wine_type_accuracy: 0.9971 - val_loss: 0.3293 - val_wine_quality_loss: 0.3026 - val_wine_type_loss: 0.0264 - val_wine_quality_root_mean_squared_error: 0.5500 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 175/180\n",
      "3155/3155 [==============================] - 0s 73us/sample - loss: 0.2707 - wine_quality_loss: 0.2555 - wine_type_loss: 0.0163 - wine_quality_root_mean_squared_error: 0.5052 - wine_type_accuracy: 0.9968 - val_loss: 0.3397 - val_wine_quality_loss: 0.3129 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5595 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 176/180\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.2708 - wine_quality_loss: 0.2560 - wine_type_loss: 0.0153 - wine_quality_root_mean_squared_error: 0.5055 - wine_type_accuracy: 0.9968 - val_loss: 0.3283 - val_wine_quality_loss: 0.3014 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5492 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 177/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2700 - wine_quality_loss: 0.2551 - wine_type_loss: 0.0152 - wine_quality_root_mean_squared_error: 0.5048 - wine_type_accuracy: 0.9968 - val_loss: 0.3296 - val_wine_quality_loss: 0.3029 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5504 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 178/180\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.2694 - wine_quality_loss: 0.2544 - wine_type_loss: 0.0152 - wine_quality_root_mean_squared_error: 0.5042 - wine_type_accuracy: 0.9968 - val_loss: 0.3263 - val_wine_quality_loss: 0.2997 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5474 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 179/180\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.2681 - wine_quality_loss: 0.2533 - wine_type_loss: 0.0151 - wine_quality_root_mean_squared_error: 0.5030 - wine_type_accuracy: 0.9968 - val_loss: 0.3264 - val_wine_quality_loss: 0.2997 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5475 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 180/180\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.2688 - wine_quality_loss: 0.2535 - wine_type_loss: 0.0150 - wine_quality_root_mean_squared_error: 0.5037 - wine_type_accuracy: 0.9968 - val_loss: 0.3270 - val_wine_quality_loss: 0.3005 - val_wine_type_loss: 0.0263 - val_wine_quality_root_mean_squared_error: 0.5481 - val_wine_type_accuracy: 0.9937\n"
     ]
    }
   ],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "history = model.fit(x=norm_train_X, y = train_Y,\n",
    "                    epochs = 180, validation_data=(norm_val_X, val_Y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "fadad8896eda9c8c2115970724b15508",
     "grade": true,
     "grade_id": "cell-eb4d5b41bef8f0ab",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_history(history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CubF2J2gSf6q"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "789/789 [==============================] - 0s 21us/sample - loss: 0.3270 - wine_quality_loss: 0.3005 - wine_type_loss: 0.0263 - wine_quality_root_mean_squared_error: 0.5481 - wine_type_accuracy: 0.9937\n",
      "\n",
      "loss: 0.3270383147081344\n",
      "wine_quality_loss: 0.3005458116531372\n",
      "wine_type_loss: 0.026257295161485672\n",
      "wine_quality_rmse: 0.5481236577033997\n",
      "wine_type_accuracy: 0.9936628937721252\n"
     ]
    }
   ],
   "source": [
    "# Gather the training metrics\n",
    "loss, wine_quality_loss, wine_type_loss, wine_quality_rmse, wine_type_accuracy = model.evaluate(x=norm_val_X, y=val_Y)\n",
    "\n",
    "print()\n",
    "print(f'loss: {loss}')\n",
    "print(f'wine_quality_loss: {wine_quality_loss}')\n",
    "print(f'wine_type_loss: {wine_type_loss}')\n",
    "print(f'wine_quality_rmse: {wine_quality_rmse}')\n",
    "print(f'wine_type_accuracy: {wine_type_accuracy}')\n",
    "\n",
    "# EXPECTED VALUES\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.018 - 0.030\n",
    "# ~ 0.50 - 0.62\n",
    "# ~ 0.97 - 1.0\n",
    "\n",
    "# Example:\n",
    "#0.3657050132751465\n",
    "#0.3463745415210724\n",
    "#0.019330406561493874\n",
    "#0.5885359048843384\n",
    "#0.9974651336669922"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gPtTGAP4usnm"
   },
   "source": [
    "## Analyze the Model Performance\n",
    "\n",
    "Note that the model has two outputs. The output at index 0 is quality and index 1 is wine type\n",
    "\n",
    "So, round the quality predictions to the nearest integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tBq9PEeAaW-Y"
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(norm_test_X)\n",
    "quality_pred = predictions[0]\n",
    "type_pred = predictions[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YLhgTR4xTIxj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.634233]\n"
     ]
    }
   ],
   "source": [
    "print(quality_pred[0])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 5.6 - 6.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MPi-eYfGTUXi"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00012555]\n",
      "[0.9999875]\n"
     ]
    }
   ],
   "source": [
    "print(type_pred[0])\n",
    "print(type_pred[944])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# A number close to zero\n",
    "# A number close to or equal to 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Kohk-9C6vt_s"
   },
   "source": [
    "### Plot Utilities\n",
    "\n",
    "We define a few utilities to visualize the model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "62gEOFUhn6aQ"
   },
   "outputs": [],
   "source": [
    "def plot_metrics(metric_name, title, ylim=5):\n",
    "    plt.title(title)\n",
    "    plt.ylim(0,ylim)\n",
    "    plt.plot(history.history[metric_name],color='blue',label=metric_name)\n",
    "    plt.plot(history.history['val_' + metric_name],color='green',label='val_' + metric_name)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6rfgSx7uz5dj"
   },
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(y_true, y_pred, title='', labels=[0,1]):\n",
    "    cm = confusion_matrix(y_true, y_pred)\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(111)\n",
    "    cax = ax.matshow(cm)\n",
    "    plt.title('Confusion matrix of the classifier')\n",
    "    fig.colorbar(cax)\n",
    "    ax.set_xticklabels([''] + labels)\n",
    "    ax.set_yticklabels([''] + labels)\n",
    "    plt.xlabel('Predicted')\n",
    "    plt.ylabel('True')\n",
    "    fmt = 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "          plt.text(j, i, format(cm[i, j], fmt),\n",
    "                  horizontalalignment=\"center\",\n",
    "                  color=\"black\" if cm[i, j] > thresh else \"white\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dfVLIqi017Vf"
   },
   "outputs": [],
   "source": [
    "def plot_diff(y_true, y_pred, title = '' ):\n",
    "    plt.scatter(y_true, y_pred)\n",
    "    plt.title(title)\n",
    "    plt.xlabel('True Values')\n",
    "    plt.ylabel('Predictions')\n",
    "    plt.axis('equal')\n",
    "    plt.axis('square')\n",
    "    plt.plot([-100, 100], [-100, 100])\n",
    "    return plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8sd1jdFbwE0I"
   },
   "source": [
    "### Plots for Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "f3MwZ5J1pOfj"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3xU9Z3/8ddnJjO5QEi4hIvcURTwAmhEXbxrFbWK2mqhtrWrW1ZXXfvrDa1bbd1qV+v2YmtV7LqKq9B2W1vqeqvX1lqpAVFBBLkJ4ZZAQkjIPfP5/XEmMsaEDJBkksn7+Xicx8x8v+fMfOYQ3uec75w5Y+6OiIikr1CqCxARkc6loBcRSXMKehGRNKegFxFJcwp6EZE0p6AXEUlzCnoRkTSnoJdexcw2mFmNmVWZ2TYze8TM+sb7HjEzN7OLWizzk3j7l+OPo2b2n2ZWHH+e9Wb24zZeo3n6eZe+UZEECnrpjS50977AFGAqcHNC32rgyuYHZpYBXAasTZjnZqAQmAbkAmcAb7X2GgnT9R3/NkSSk5HqAkRSxd23mdlzBIHf7I/AF8ysv7uXAzOAdwgCvdnxwJPuviX+eEN8EumWtEcvvZaZjQDOA9YkNNcCi4BZ8cdfAua3WPQN4Gtm9i9mdrSZWacXK3IQFPTSG/3ezCqBTUAJcFuL/vnAl8wsDzgN+H2L/h8AdwFXAEXAZjO7ssU8vzezXQnTVzr8XYgkSUEvvdHF7p4LnA5MAAYldrr7a0AB8G/AU+5e06K/yd3vc/fpQD5wB/CwmU1s8Rr5CdNDnfh+RPZJQS+9lru/CjwC3NNK9/8AX+eTwzYtn6PG3e8DyoFJHV2jSEfQh7HS2/0E2GBmU1q03wv8BfhzywXM7KvAMmAx0EAwhJPLJ8+8EekWFPTSq7l7qZnNB74DVCa0lwEvtrFYDfCfwGGAE5yS+Rl3X5cwzx/NrCnh8Z/c/ZIOLV4kSaYfHhERSW8aoxcRSXPtBr2ZjTSzl81spZmtMLMbW5nHzOxeM1tjZu+Y2bEJfTPMbFW876aOfgMiIrJvyezRNwJfd/eJwInAdWbW8uyC84Dx8WkOcD+AmYWB++L9k4DZrSwrIiKdqN2gd/et7r40fr8SWAkMbzHbTGC+B94A8s1sGMG1QNa4+zp3rwcWxucVEZEusl9n3ZjZGIKLQC1u0TWc4FuGzYrjba21n9DGc88hOBqgT58+x02YMGF/SvuY9bvWU161h77VR3H44Qf8NCIiPcaSJUt2uHtBa31JB338Uq6/Bb7q7rtbdreyiO+j/ZON7vOAeQCFhYVeVFSUbGmf8OXff5mFb7zM8W8W8WJbJ8iJiKQRM/uwrb6kgt7MIgQh/7i7/66VWYqBkQmPRwBbgGgb7Z0qGo7ioQaamtqfV0Qk3SVz1o0B/wWsdPcftTHbIoKLQJmZnQhUuPtW4E1gvJmNNbMowRUBF3VQ7W2KhCJ4qF5BLyJCcnv004EvAu+a2bJ427eBUQDu/gDwNHA+weVeq4F/jPc1mtn1wHNAGHjY3Vd06DtoRTQcJWbaoxcRgSSCPn4lv31eb9uDr9de10bf0wQbgi4TCUdwBb2ICJCm34zV0I2IyF5pGfTRcBS3JhoaY6kuRUQk5dIy6CPhCACN3pDiSkREUi8tgz4ajgLQpKAXEUnPoI+E4nv0MQW9iEh6Bn3z0A31Ka5ERCT10jLoNXQjIrJXWgZ989BNk/boRUTSM+ib9+g1Ri8ikqZB3zxG34SCXkQkPYNeQzciIh9Jy6D/6MNY7dGLiKRn0H90emVMe/QiImkZ9M179LUNDXirv2clItJ7pGXQN4/RE2qgtja1tYiIpFp6Bn186IZwPXv2pLYWEZFUS8ugbx66IdygoBeRXi8tg/6joRvt0YuItP9Tgmb2MPBpoMTdj2ql/5vAFQnPNxEocPcyM9sAVAJNQKO7F3ZU4fvy0R59SHv0IiLJ7NE/Asxoq9Pdf+juU9x9CnAz8Kq7lyXMcka8v0tCHhLH6BX0IiLtBr27/xkoa2++uNnAgoOqqAPsHaPX0I2ISIeN0ZtZDsGe/28Tmh143syWmNmcjnqt9iSeXqmgF5Hert0x+v1wIfDXFsM20919i5kNBv5kZu/HjxA+Ib4hmAMwatSogyok8fTKqqqDeioRkR6vI8+6mUWLYRt33xK/LQGeBKa1tbC7z3P3QncvLCgoOKhCdHqliMheHRL0ZpYHnAb8IaGtj5nlNt8HzgGWd8TrtUdDNyIieyVzeuUC4HRgkJkVA7cBEQB3fyA+2yXA8+6eGKtDgCfNrPl1nnD3Zzuu9LaFQ2FCFiKWoQ9jRUTaDXp3n53EPI8QnIaZ2LYOmHyghR2sSChCLFN79CIiHflhbLcSCUeIZWmPXkQkLS+BAMEHshnaoxcRSeM9+lAEog3s2Z3qSkREUittgz4ajhKLauhGRCRth24i4QjhiIZuRETSN+hDEUJRfTNWRCRtgz4ajhLK0B69iEjaBn0kHFHQi4iQxkEfDUdB34wVEUnfoI+EIlj8ombuqa5GRCR10jfowxEI1xOLQV1dqqsREUmdtA36aDgKoQYADd+ISK+WtkEfCUVwBb2ISPoGfTQcJWb1gIJeRHq3tA36SDiCm/boRUTSNuij4ShNBHv0+nasiPRmaRv0kVCEJrRHLyKioBcRSXNpG/TRcJRG14exIiLtBr2ZPWxmJWa2vI3+082swsyWxadbE/pmmNkqM1tjZjd1ZOHtGZA9gMr6CgjrMggi0rsls0f/CDCjnXn+4u5T4tPtAGYWBu4DzgMmAbPNbNLBFLs/RuWNwnHI3aygF5Ferd2gd/c/A2UH8NzTgDXuvs7d64GFwMwDeJ4DMipvVHAnf6OCXkR6tY4aoz/JzN42s2fM7Mh423BgU8I8xfG2VpnZHDMrMrOi0tLSgy6oOeijgz9U0ItIr9YRQb8UGO3uk4GfAb+Pt1sr87Z5HUl3n+fuhe5eWFBQcNBFjeg3AoDIQO3Ri0jvdtBB7+673b0qfv9pIGJmgwj24EcmzDoC2HKwr5es7Eg2g/sMJjRgI7t3d9Wrioh0Pwcd9GY21Mwsfn9a/Dl3Am8C481srJlFgVnAooN9vf0xOm80kYEb2bChK19VRKR7yWhvBjNbAJwODDKzYuA2IALg7g8AnwWuNbNGoAaY5e4ONJrZ9cBzQBh42N1XdMq7aMOovFF8kPsea9Z05auKiHQv7Qa9u89up//nwM/b6HsaePrASjt4o/JGUR15lvpSp6LCyMtLVSUiIqmTtt+MhSDo69kD2WXaqxeRXivtgx6AvI0KehHptXpN0H/wQWprERFJlV4R9HmjtUcvIr1XWgd9QU4BWRlZ5I5Q0ItI75XWQW9mjMkfQ7hgrYJeRHqttA56gKMGH0VVzjts346+ISsivVLaB/3kIZPZ6WshWsnatamuRkSk66V90B8z5JjgzuDlrFqV2lpERFIh7YN+8pDJAGSMeJslS1JcjIhICrR7CYSeblTeKPIy88ia9A5//3uqqxER6Xppv0dvZhwz5BhCw96mqAgaG1NdkYhI10r7oIdgnL48+i7VNTFWrkx1NSIiXatXBP3kIZOp9UrI36DhGxHpdXpH0A8NPpDNOfQtBb2I9Dq9I+iHTCY7I5tBx72qoBeRXqdXBH1mRianjD6F6qEv8u67UFmZ6opERLpOrwh6gLPGnsUOe4+m7K289FKqqxER6TrtBr2ZPWxmJWa2vI3+K8zsnfj0uplNTujbYGbvmtkyMyvqyML311ljzwIgc+JLPPNMKisREelayezRPwLM2Ef/euA0dz8G+HdgXov+M9x9irsXHliJHWPK0Cn0z+rPkBNf5JlnwD2V1YiIdJ12g97d/wyU7aP/dXcvjz98AxjRQbV1qHAozBljz6Cy4AU2bnSdTy8ivUZHj9FfDSQOjDjwvJktMbM5+1rQzOaYWZGZFZWWlnZwWYFzxp1DeWwTDF6u4RsR6TU6LOjN7AyCoJ+b0Dzd3Y8FzgOuM7NT21re3ee5e6G7FxYUFHRUWR8zc8JMDGPQab/lqac65SVERLqdDgl6MzsG+CUw0913Nre7+5b4bQnwJDCtI17vQA3tO5STR51MaNJvefVVKC5OZTUiIl3joIPezEYBvwO+6O6rE9r7mFlu833gHKDVM3e60mcnfZYSW44PWMXjj6e6GhGRzpfM6ZULgL8BR5hZsZldbWbXmNk18VluBQYCv2hxGuUQ4DUzexv4O/B/7v5sJ7yH/XLpxEsBGHnub3nsMZ19IyLpz7wbJl1hYaEXFXXeafcn/ddJFG+rofjflvHWWzBlSqe9lIhIlzCzJW2dxt5rvhmbaPZRsylufJuMYSt47LFUVyMi0rl6ZdB/7sjPEbYw4y5+nCee0I+RiEh665VBP6TvEM4edzblI55g2/YYL76Y6opERDpPrwx6gCuOvoLShg/pO/F1Dd+ISFrrtUF/8YSLyc7IZuQFj/Pkk7p0sYikr14b9LmZuVw84WI25/+a6rp65s9PdUUiIp2j1wY9BMM3uxvKOGzGs9x7L8Riqa5IRKTj9eqgP+fQcxiUM4iBZzzO6tXw3HOprkhEpOP16qCPhCNcPuly3q5dxJDRFfz0p6muSESk4/XqoAe4aupV1DbWMuWq/+K55+D991NdkYhIx+r1QX/cIcdx2ujTWN7nJ0SyGrj33lRXJCLSsXp90AN84x++weaqTZx09W949FEoL29/GRGRnkJBD5w//nwmDprI1kPvoromxi9/meqKREQ6joIeCFmIb5/ybT7Y/Q5HXva//PjHUFeX6qpERDqGgj5u9lGzObLgSHYfdytbtzfy6KOprkhEpGMo6OPCoTD/fsa/s6lmFWNmPsZdd+mqliKSHhT0CS6ecDGFhxSyp/C7rPuwjieeSHVFIiIHT0GfwMy488w7KW3YyIiZD/G970FDQ6qrEhE5OMn8ZuzDZlZiZq3+sLcF7jWzNWb2jpkdm9A3w8xWxftu6sjCO8vZ487mtNGnUXXs91m3aQ///d+prkhE5OAks0f/CDBjH/3nAePj0xzgfgAzCwP3xfsnAbPNbNLBFNsVzIw7zryDXY3bGXXZz7j9dqiuTnVVIiIHrt2gd/c/A2X7mGUmMN8DbwD5ZjYMmAascfd17l4PLIzP2+1NHzWdC8ZfQPmku9m8cxd3353qikREDlxHjNEPBzYlPC6Ot7XV3iozm2NmRWZWVFpa2gFlHZzvn/l9KhvLmfRP/8ndd8PGjamuSETkwHRE0Fsrbb6P9la5+zx3L3T3woKCgg4o6+BMGTqFy4+8nA1Df0wsu4S5c1NdkYjIgemIoC8GRiY8HgFs2Ud7j/G9079HbVMNU274DxYuhNdeS3VFIiL7ryOCfhHwpfjZNycCFe6+FXgTGG9mY80sCsyKz9tjTBg0gSsnX8myjF8w5PBN3HijfoVKRHqeZE6vXAD8DTjCzIrN7Gozu8bMronP8jSwDlgDPAT8C4C7NwLXA88BK4Ffu/uKTngPneq2024j5jEmXfPvLF0KDz2U6opERPaPubc5bJ4yhYWFXlRUlOoyPnLD0zdwf9H9FL7xPiv/ehgrV8Ihh6S6KhGRvcxsibsXttanb8Ym4ZZTbyEajjL4c7dRXw/XX5/qikREkqegT8LQvkO58YQbeWrDAq657V2efBJ+97tUVyUikhwFfZK+Of2b5Gbmsm70d5g6Fa67DnbtSnVVIiLtU9AnaUD2AL75D99k0eo/cOMPF1NaCt/6VqqrEhFpn4J+P9x4wo0MyhnE/M3f5v99zXnoIXjllVRXJSKybwr6/ZCbmcutp97KS+tfInTWdxg3DubMgZqaVFcmItI2Bf1+un7a9cw5dg53v3EH5333Z3zwAXzve6muSkSkbQr6/WRm/OKCX3DRERfx0Iff4DNzVnHPPbBkSaorExFpnYL+AIRDYeZ9eh45kRy2T7uGgsHOVVdBXV2qKxMR+SQF/QEa0ncId599N68Vv8Lldz7CO+/A176W6qpERD5JQX8Qrj72ak4edTKPbf86136zhF/8AubPT3VVIiIfp6A/CCELMe/T86iqr2LXCV/j9NPh2mth1apUVyYispeC/iBNLJjIzSffzILljzPr9t+QnQ2f/zzU16e6MhGRgIK+A9xy6i2cNOIkvvHaVXz3vpUsXYp+kUpEug0FfQeIhqP8+rJfk52RzS92fIZr/rWKn/wEFixIdWUiIgr6DjOi3wgWfnYhq3auYuf0f+LkU5yrr4alS1NdmYj0dgr6DnTm2DO548w7+M3KX3HYDdcz8JDdnH8+rFuX6spEpDdT0HewudPncsO0G3j0vfup/coR1PZ9n3PPhdLSVFcmIr1VUkFvZjPMbJWZrTGzm1rp/6aZLYtPy82sycwGxPs2mNm78b7u8/uAncTMuPe8e1n8T4uJWT0jr7+KTZubuOACqKpKdXUi0hsl8+PgYeA+4DxgEjDbzCYlzuPuP3T3Ke4+BbgZeNXdyxJmOSPe3+rvGaaj44cfz09n/JTlFX/jC/f+jCVL4PLLoaEh1ZWJSG+TzB79NGCNu69z93pgITBzH/PPBnS+CXDF0VdwwfgLeGzbXG74yfM880xwWeNu+HvsIpLGkgn64cCmhMfF8bZPMLMcYAbw24RmB543syVmNqetFzGzOWZWZGZFpWkyoG1mzL9kPhMHTeTB3TO58rZXeeQR+Jd/gVgs1dWJSG+RTNBbK21t7ZNeCPy1xbDNdHc/lmDo5zozO7W1Bd19nrsXunthQUFBEmX1DAOyB/CnL/6JsfljeSrnM1xz04c88AB84QsaxhGRrpFM0BcDIxMejwC2tDHvLFoM27j7lvhtCfAkwVBQr1LQp4A/zPoDDbEGloy7jO//oI4FC+Dii6G6OtXViUi6Sybo3wTGm9lYM4sShPmiljOZWR5wGvCHhLY+ZpbbfB84B1jeEYX3NOMHjueRmY/w5pY3+evIS/jZAzU88wycdRZs3Zrq6kQknbUb9O7eCFwPPAesBH7t7ivM7BozuyZh1kuA5919T0LbEOA1M3sb+Dvwf+7+bMeV37NcMvESHrrwIZ5d8yy/jp7Lvf/zAe+8A8cdB0Vpf+KpiKSKeTc8BaSwsNCL0jj5Fi5fyFf++BXqGuu46vCbefam2ygvC/Hcc3DiiamuTkR6IjNb0tYp7PpmbArMOmoWH9zwAZ876nM8+P7tTP7eFQwcXMc558DChamuTkTSjYI+RYb2Hcr8i+dz19l3sWj9Qgq+/imOmFLG7Nlw5ZVQWZnqCkUkXSjoU8jM+Nb0b7HwMwtZtmMxZZ85num3fZvHXnqTKVPg9ddTXaGIpAMFfTfwuaM+xwtffIFBfQaxOPxDQl85iYpDf8nJJ8P/+3+wZ0/7zyEi0hYFfTdxyuhTWPxPi9nxzR186tBPsXP6VzjuG9/hJz9xjjkGXnop1RWKSE+loO9m8rLyWDRrEVdPvZqiPt/nnPuvxLPKOeus4LdoN21q/zlERBIp6LuhSDjCQxc+xO2n387z2x9j+xUjOPa2f+Z//7ycceOCyycs75VfOxORA6Gg76bMjO+c9h2W/fMyZh81m/ci82n4ytEMnzuD371RxNFHB5c9VuCLSHv0hakeYmf1TuYtmcc9f7uHspoyBvjhVL51Lg0vz+W8k4dz+eVw0UUwYECqKxWRVNjXF6YU9D3M7rrdPPzWw7y4/kWeX/M83pRB5qovULVqGuG1F3H2SQV89rPBBdMGDUp1tSLSVRT0aWpd+TpueekW/m/1/1FZX0nYM8le+3mqVp5AqGIchZMGc8W5E7j80kyGDk11tSLSmRT0aS7mMZaXLOfnf/85C5YvoKo+4cdpa/Ng1YXkNU7g8PyjuHjipznl5DCFhZCdnbqaRaRjKeh7kZjH2FSxiQ8rPmRL5VYWFD3Dix8+zR7iv9pVOgF2j4RD3iQcChENZRPKaGBEzuFcO/nrfOGE88jvl8FL619iT8Mezh9/PtFwtN3X3Va1DXdnWO6wTn6HHcvdMWvtt3VEehYFvVDbWMtTq5/ijlfvYldlPf2rTqJ8ZwY7d9dQWZEB4/4E/ddDLIzV5+JZuwDIaRrGkPBEwtE6QpF68nP6csa409hcvYZn1jzDeePP4/hDjuffXvo3Yh7jzrPuJDOcyYrSFXxp8pfIy8zjntfvYUD2AC6ZeAnbq7azuXIz2RnZDO07lMMHHk6/zH70jfYlMyOT1za+xs0v3sxlky7jhmk3tBvCWyq38MK6Fzhz7JmM6Ddiv9bJjuodnPs/59Ivsx+PX/o4meFM3tzyJu7O0UOO3q/nq6yr5K1tb1F4SCE5kZx253d3Vu1cxREDjzioDU1ZTRkDsvUJvCjopR21tfDe+408UfQ0S7ctZmvVVvpuPZ/ykmw2DX6I+vBOaIoGU9/tMHQZ1OfSZ/tZVA9/Bg/XMqz2dLKiUdaHngcgTAZNNGKEiFgmTdTT5E1t1mAYI/NGsrFiI30ifdjTsIfPH/15pg6dSvHuYoq2FDEwZyBTh05lTdka1pStYU/DHt4rfY+Yx8jKyOIfp/wjRww8goI+BQzIHsD68vUs3bqUpduWEglFuO7467joiIvol9mP9bvWc9lvLuO90vcIW5hwKMye+j0f1ZgTyeHeGffy5Slfpqaxhlc2vMLasrWEQ2EyQhlkZWQxfeR06prquO2V23hq9VPUN9VzaP9D+fG5P+awAYdR0KeAgdkDMTPqGuv44+o/Ury7mGOGHMM9r9/DM2ue4aopV/HghQ+SEcqgZE8Jb20NNhYDcwa2up6a/782eRNz/zSXH73xI26afhN3nnXnRxsMd2fT7k1sqdzC1KFTCVmIt7a9xbj+4xiUc2Cf0Ls7Wyq3MDBnII2xRu78y52sK1/HtYXXkpuZy1tb32JM/himDJ3yUe31TfUfOxp8ef3LzH1hLhcefiG3nHoLIfv42d27andRuqeUxlgjY/uPJSsj62P9G3ZtoLqhmgmDJhCy0Efrovl91zbW8sqGVzhxxInkZ+WzvWo70XCU/tn9P/Y8bxS/wR1/uYOzx57Nv57wr21uaMtqyrjphZvIz8rn9jNuJ2xhNlZsZGz/sZ+ovTWNsUZ2Vu9kSN8hLC5ezA9e+wGXTryULx7zxU45ilTQy0Gpq4MtW2Dz5mBaU7yLki1ZlGzJYmdDMcWxJZS+diHl5YaPeYnY7iFQMQqOexCydsHifwVzGPMy7BoDFaOJ5tTSd/gmMoetIdp3D+E+ZdTnrqY/4zixaS4r8u/ibxk/wIkRtWzGZk2l0kvYUruGIdkjOCx/ArmZORw56BhOHnYuC1bP48kPFtIQ+/gP8Q7MHsixw46leHcxK3esBCA7I5uaxhoioQh/mPUHxuSP4bZXbuOwAYdx7qHnErIQt71yGy9veBkINkLe5s8kQ7/Mflw99WqmDJ3C7a/eztrytR/19Y32JS8zj6r6KirqKj5qz4nkcP748/nf9/6XowYfRXVDNevK1wGQlZHF+ePPJzOcycaKjSwvWU6/zH4M7TuUteVrqaqvIi8zj9LqUo4ddixLty7lgvEX0C+zH5t2b2L1ztWU7CkBoE+kD+FQmN11u8kIZXDyqJOJhCLUNNZQ01BDdUM1DbEG8rPyGdxnMAU5BWys2MjfN/+drIws+mf3JyOUwbaqbeyq3UU0HKVvtC9lNWX0z+pPeW35J9bHyH4jcZzi3cVMHDSRacOnsa58HX/Z+Bfys/LZVbuLU0efSkVtBaXVpRxZcOTH/n2a1/mA7AHkZeWRn5VPdUM17+94H4D+Wf0Jh8LsqN4BwOA+g5lx2AxeXv8ym3Zvom+0L1OGTuGvG/9KJBzh4gkXE7IQG3ZtoKK2gpU7Vn70N3D6mNM5YuARREIRBvcZzPs732dx8WKG5Q5jbdlaSvaU0ORNTBg0gbKaMkr2lDCkzxDOOfQczhx7JstLlvPqh6+SE8lhYPZABmQPYED2AGIeY8HyBWyp3MLw3OFsqdxCJByhvqmeacOnkZWRRXZGNlOHTqVvtC/1TfXUN9WTHcnm1tNuTfr/ZiIFvXQZd9i2DYqLIS8PIhEoL4eysr23iffLy2H3bqiqCi7NnDg1UgOhJmjMglhG8AIZNdDYxqfIFoOsXWTklRDqs5P80EiGZI8kI2xk58QIH/YysSFLifXZTP/YEYy1MzhqyAQyMoLXy84Oas7Lg2hWEy+XPsG2uvVgMU4dfQpThk6hsspxayQW3UXRzpeo8V1ce/w/f7QXW91QzSsbXqGitoLte7azYdcGKusqyQhlcOnESzlq8FEUbSliytApjM4fzf1v3s+jbz/KmPwxTB06lclDJ/P793/PC+teIGQhCvoUMHnIZCrrK9lauZVD+x9KflY+mys3c8H4C5h11Cy+8/J3eHDJg+Rl5nFI7iGMHzCeqcOmMqzvMF5a/xKNsUZOH3M6b29/mxfXv0hGKIPsjGyyI9nkRHKIhCKU15ZTsqeEkj0lFOQU8A8j/wF3p6y2jJjH6J/VnyMLjmTT7uDzn6+e8FWmDJ3Cr1b8ioxQBtOGT+PDXR+ybNsylm1fhmGMyR/D4s2LeWf7Oxza/1A+Ne5TzD15Lg8WPch//PU/OLLgSEb0G8GK0hUU5BRw6uhTGZU3CsNYW76WbVXbqKiroKI22ECeNfYs8rPy+Vvx38gIZTAoZxAhC7Fq5yqe/uBpJgyawFdP+Cp/XP1Hlpcs5+IJF1NeU87CFQvJy8xjTP4Y8rLymDp0KjeecCPz357PHX+5g5jHqGuqY1ftLgb3Gcz0kdPZUb2DkIX40bk/YvPuzXzrhW8xYdAEzhp7Fq9vep3n1z7PzpqdREIRpo+aTsxjlNWUUVZTxs7qndQ31TPjsBmcPuZ0lmxdwqH9D2Xu9LnMf3s+/73sv8nNzKWitoIVpStojDUCEA1HGZ47nHU3rjug/3sKeumR6uqCAG65EUh8DBCNQn198EPr1dXB/cbGYEOyYwfEYsEyW7cGRyYdfa3/rCzIzYW+fYOp+X5TU7AhC4ehTx/Iydl72zxlZgZTNLr3NvF+e32RCIRCwe3AgcFtb9QRH6rXNtYSDUeTGpZpijWxonQFI/uN/MTQEATDNhmhjHafp6GpAceJhCIHXWsNSdIAAArwSURBVP++gr79SoInmAH8FAgDv3T3/2jRfzrBj4Kvjzf9zt1vT2ZZkbY0h2BHf/Grrg4yMoINQElJcBTSpw/U1EBFRTDV1gbtsViw4di5M1guNzcI8OaNTVXV3inxcUVFEMBDhwbPsWcPbN8e3FZXB6+1Z8/ejVJHycoK3ltbUySy9zYrKziKycwM2kKhYAqH996PRD65AWu+zcgI3ks4vHfDFYlAQ0PwvIMGBc/R0BC8x+zsYH00NQXvPScneJ5oFA52yLojxrxbfiawL+FQmGOGHNNmfzIhD8F1rbpCu9WYWRi4D/gUUAy8aWaL3P29FrP+xd0/fYDLinSZzMzgNhyG4cP3tvfvD4cc0vX1NG9M6uqC28T77d3W1wcbpLo6KC0Ngrexsf2pri7YmNXWwq5dQfjGYntvm+83NOzdeNXVdc77z8gIAr9Pn+B1m2tsaAhqyM6G/HwYPz4YVquoCN5z4gYsHP7442h078YnOzvoN9u7QWm+H4nAyJHBRqmuLpgaGqCgIPh7qKoKXisra++GMSsreL5YbO9RWnc/QzeZzc40YI27rwMws4XATCCZsD6YZUV6hVBob5B0Z82h33zk0tCwN5wTh80ikeCIpTT+1Y3mo4g9e4LPbzIy9h5BNQ/B7dkTTC0DOxwONkY7dsDq1bB+fRD64fDHNwgtN2T19XuPmmKxzl0viSGfOBKenb33aCgcDupsaAjmyc2Ffv323ubkBOsvOxsWLOj4GpMJ+uFA4lXQi4ETWpnvJDN7G9gCfMPdV+zHspjZHGAOwKhRo5IoS0S6UiQS7OX2/+SQdLflvvfIoDmE3fdOdXWwcWPweU5WVnC0Fw4HQ3oVFXtDurY22HA0HwU1NgYb6Orq4GQC+PgRg3swf/PRUCy2d8gM9m4wd+8ONl7V1UHYd9YRZTJB39pBSctPcJcCo929yszOB34PjE9y2aDRfR4wD4IPY5OoS0Rkn8yCYZy25Ob2jov/JXM9+mJgZMLjEQR77R9x993uXhW//zQQMbNBySwrIiKdK5mgfxMYb2ZjzSwKzAIWJc5gZkMt/rG3mU2LP+/OZJYVEZHO1e7Qjbs3mtn1wHMEp0g+7O4rzOyaeP8DwGeBa82sEagBZnlwgn6ry3bSexERkVboC1MiImlgX1+Y0m/GioikOQW9iEiaU9CLiKQ5Bb2ISJpT0IuIpDkFvYhImlPQi4ikOQW9iEiaU9CLiKQ5Bb2ISJpT0IuIpDkFvYhImlPQi4ikOQW9iEiaU9CLiKQ5Bb2ISJpT0IuIpDkFvYhImksq6M1shpmtMrM1ZnZTK/1XmNk78el1M5uc0LfBzN41s2Vmpt8HFBHpYu3+OLiZhYH7gE8BxcCbZrbI3d9LmG09cJq7l5vZecA84ISE/jPcfUcH1i0iIklKZo9+GrDG3de5ez2wEJiZOIO7v+7u5fGHbwAjOrZMERE5UMkE/XBgU8Lj4nhbW64Gnkl47MDzZrbEzObsf4kiInIw2h26AayVNm91RrMzCIL+5ITm6e6+xcwGA38ys/fd/c+tLDsHmAMwatSoJMoSEZFkJLNHXwyMTHg8AtjSciYzOwb4JTDT3Xc2t7v7lvhtCfAkwVDQJ7j7PHcvdPfCgoKC5N+BiIjsUzJB/yYw3szGmlkUmAUsSpzBzEYBvwO+6O6rE9r7mFlu833gHGB5RxUvIiLta3foxt0bzex64DkgDDzs7ivM7Jp4/wPArcBA4BdmBtDo7oXAEODJeFsG8IS7P9sp70RERFpl7q0Ot6dUYWGhFxXplHsRkWSZ2ZL4DvYn6JuxIiJpTkEvIpLmFPQiImlOQS8ikuYU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikuYU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikuYU9CIiaU5BLyKS5hT0IiJpLqmgN7MZZrbKzNaY2U2t9JuZ3Rvvf8fMjk12WRER6VztBr2ZhYH7gPOAScBsM5vUYrbzgPHxaQ5w/34sKyIinSiZPfppwBp3X+fu9cBCYGaLeWYC8z3wBpBvZsOSXFZERDpRRhLzDAc2JTwuBk5IYp7hSS4LgJnNITgaAKgys1VJ1NaaQcCOA1y2K/WUOqHn1NpT6oSeU2tPqRN6Tq2dVefotjqSCXprpc2TnCeZZYNG93nAvCTq2SczK3L3woN9ns7WU+qEnlNrT6kTek6tPaVO6Dm1pqLOZIK+GBiZ8HgEsCXJeaJJLCsiIp0omTH6N4HxZjbWzKLALGBRi3kWAV+Kn31zIlDh7luTXFZERDpRu3v07t5oZtcDzwFh4GF3X2Fm18T7HwCeBs4H1gDVwD/ua9lOeSd7HfTwTxfpKXVCz6m1p9QJPafWnlIn9Jxau7xOc291yFxERNKEvhkrIpLmFPQiImkubYK+O19qwcxGmtnLZrbSzFaY2Y3x9u+a2WYzWxafzu8GtW4ws3fj9RTF2waY2Z/M7IP4bf9uUOcRCettmZntNrOvdod1amYPm1mJmS1PaGtzHZrZzfG/21Vmdm43qPWHZvZ+/HImT5pZfrx9jJnVJKzbB1JcZ5v/1t1wnf4qoc4NZrYs3t4169Tde/xE8EHvWmAcwSmdbwOTUl1XQn3DgGPj93OB1QSXhPgu8I1U19ei1g3AoBZtdwM3xe/fBNyV6jpb+fffRvCFkZSvU+BU4FhgeXvrMP538DaQCYyN/x2HU1zrOUBG/P5dCbWOSZyvG6zTVv+tu+M6bdH/n8CtXblO02WPvltfasHdt7r70vj9SmAlwbeGe4qZwKPx+48CF6ewltacBax19w9TXQiAu/8ZKGvR3NY6nAksdPc6d19PcObatC4plNZrdffn3b0x/vANgu+/pFQb67Qt3W6dNjMzAy4HFnRVPZA+QzdtXYKh2zGzMcBUYHG86fr4IfLD3WFIhOCby8+b2ZL4ZSkAhnjwvQjit4NTVl3rZvHx/zjdbZ1C2+uwu//tXgU8k/B4rJm9ZWavmtkpqSoqQWv/1t15nZ4CbHf3DxLaOn2dpkvQJ32phVQys77Ab4Gvuvtugqt8HgpMAbYSHNKl2nR3P5bgiqPXmdmpqS5oX+JfxLsI+E28qTuu033ptn+7ZnYL0Ag8Hm/aCoxy96nA14AnzKxfquqj7X/rbrtOgdl8fKekS9ZpugR9MpdpSCkzixCE/OPu/jsAd9/u7k3uHgMeogsPL9vi7lvityXAkwQ1bbfgaqTEb0tSV+EnnAcsdfft0D3XaVxb67Bb/u2a2ZXAp4ErPD6YHB8K2Rm/v4Rg7PvwVNW4j3/r7rpOM4BLgV81t3XVOk2XoO/Wl1qIj8v9F7DS3X+U0D4sYbZLgOUtl+1KZtbHzHKb7xN8KLecYF1eGZ/tSuAPqamwVR/bQ+pu6zRBW+twETDLzDLNbCzBbzr8PQX1fcTMZgBzgYvcvTqhvcCC35jAzMYR1LouNVXu89+6263TuLOB9929uLmhy9ZpV30S3dkTwSUYVhNsEW9JdT0tajuZ4NDxHWBZfDofeAx4N96+CBiW4jrHEZyt8Dawonk9AgOBF4EP4rcDUr1O43XlADuBvIS2lK9Tgg3PVqCBYO/y6n2tQ+CW+N/tKuC8blDrGoIx7ua/1Qfi834m/nfxNrAUuDDFdbb5b93d1mm8/RHgmhbzdsk61SUQRETSXLoM3YiISBsU9CIiaU5BLyKS5hT0IiJpTkEvIpLmFPQiImlOQS8ikub+P+Aj4QNO/G5tAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_quality_root_mean_squared_error', 'RMSE', ylim=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QIAxEezCppnd"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3hc9X3n8fdHM6OrJV9lMLIdDDEQyBIujkmTQNKyFEMKJk2TwuZCm3QJ3VBK02zLNmmXZ9s8m3ubplkoKTxr0iQkaZKNuyUFSrMQkkAsCHfHYIwJso1tfJNtWbIu3/3jnLHHw0iasSWNpPm8nuc8c87v3H5zLM9nfr9zGUUEZmZWe+qqXQEzM6sOB4CZWY1yAJiZ1SgHgJlZjXIAmJnVKAeAmVmNcgDYlCbpfEnrql0Ps6nIAWCTiqT/JumuorLnhim7MiJ+FBGnjnEdFkvaVzCEpP0F0+eP5f6GqcNNkv5xvPdjtS1b7QqYFXkAuFFSJiIGJR0P5IBzispemy475iLil8CM/LSkAN4QEevHY39m1eIWgE02a0g+8M9Kpy8AfgisKyp7PiI2S3q7pK78ypI2SvqYpCck7ZH0TUmNBfN/Q9JjknZL+omkM8utmKTjJfVImltQdq6k7ZJykn5H0o8lfSnd9y8kXViw7ExJt0naImmTpL+SlKn0AEm6XNLT6Xv4f5JeVzDvT9Nt75W0Lr9/ScsldUrqlrRV0hcq3a9NPw4Am1Qi4iDwMMmHPOnrj4AHi8pG+vb/HmAFsAQ4E/gdAEnnALcDHwbmAn8PrJbUUGbdXgb+X7r9vPcBd0ZEfzp9HrABmAf8d+C7kuak81YBAyStl7OBXwd+r5x950k6BfgGcAPQDtwF/LOkekmnAtcBb4yIVuBiYGO66heBL0ZEG3Ay8K1K9mvTkwPAJqP7Ofxhfz5JAPyoqOz+Edb/24jYHBE7gX/mcMvhPwN/HxEPR8RgRKwC+oA3VVC3VSQf+qTf3q8CvlowfxvwNxHRHxHfJGm5vEPSccAlwA0RsT8itgF/DVxZwb4Bfhv4l4i4Nw2dzwFNwJuBQaABOF1SLiI2RsTz6Xr9wGslzYuIfRHxUIX7tWnIAWCT0QPAWyXNBtoj4jngJ8Cb07LXM3IL4OWC8R4O9+e/BvjjtOtkt6TdwCLghArq9n2SD9iTgIuAPRHxs4L5m+LIJyy+mG7/NSRdW1sK9v33wPwK9k26rRfzExExBLwEdKTnKG4AbgK2SbpTUv69fQg4BfiFpDWSfqPC/do05ACwyeinwEzgGuDHABHRDWxOyzZHxAtHsd2XgE9GxKyCoTkivlHuBiKil6T75L3A+zny2z9AhyQVTC9O6/0SSWtjXsG+2yLijArfw2aSMAEg3dciYFNav69HxFvTZQL4dFr+XERcRRI4nwb+SVJLhfu2acYBYJNORBwAOoGPknT95D2Ylh3t1T9fAa6VdJ4SLZLeIam1wu3cQXJe4XKg+FLN+cD16UnhdwOvA+6KiC3APcDnJbVJqpN0sqS3jbCfOkmNBUMDSfi8Q9KFknLAH5MEy08knSrp19LleoEDJN1CSHqfpPa0xbA73f5ghe/bphkHgE1W95N8mD5YUPajtOyoAiAiOknOA/wdsAtYT3qCuMLt/BgYAh6NiI1Fsx8GlgKvAJ8EfisidqTzPgDUA8+k+/8nYMEIu7qK5EM8PzwfEetIzkF8Kd3HZcBl6cnzBuBTafnLJMfqz9JtrQCelrSP5ITwlWlrxmqY/IMwZpWT9O/A1yPiHwrKfgf4vbQLxmzS841gZhWS9EbgHGBltetidizK6gKStCK9qWS9pBtLzH9veuPNE+nNNW8YbV1JcyTdq+SW/nvTqzvMJjVJq4B/I7mcc2+162N2LEbtAkqvdX6W5JK3LpI7Na+KiGcKlnkzsDYidkm6BLgpIs4baV1JnwF2RsSn0mCYHRF/Og7v0czMSiinBbAcWB8RG9ITTXdS1PSNiJ9ExK508iFgYRnrriS5qYb09YqjfxtmZlapcs4BdJBcw5zXRXK7+3A+BPygjHWPSy+NIyK2SCp5Q4yka0iu/aalpeXc0047rYwqV2bLvi1s7t5M855zeN3rNPoKZmZTyCOPPPJKRLQXl5cTAKU+EUv2G0n6VZIAyF8FUfa6w4mIW4FbAZYtWxadnZ2VrF6Wz/74s/zJv/0JJ33/ATo7fW+MmU0vkl4sVV5OF1AXyZ2GeQtJ7kYs3sGZwD8AKwuuex5p3a2SFqTrLiB5hkpVNGSTZ4Ed6O+rVhXMzCZcOQGwBlgqaYmkepKHV60uXEDSYuC7wPsj4tky110NXJ2OX03yjJWqqM/UA9Dbf7BaVTAzm3CjdgFFxICk64C7gQxwe0Q8LenadP4twF+QPF73f6WPQRmIiGXDrZtu+lPAtyR9CPgl8O4xfm9la8gkLYBetwDMrIaUdSNYRNxF8tzxwrJbCsZ/j2Gea15q3bR8B3Dhq9eYeIdaAANuAZhZ7fCzgDh8DqBv0C0AM6sdDgAOtwAGhg4yNFTlypiZTRAHAIfPAZDpo8+NADOrEQ4ADrcAyByk1w/INbMa4QDg8DkAsn0OADOrGQ4A3AIws9rkAODIcwAOADOrFQ4A3AIws9rkAMDnAMysNjkAcAvAzGqTA4CCcwDZXgeAmdUMBwDQnGtORrIHHABmVjMcAEBjtjEZyTkAzKx2OAAASTRmmiDX40dBmFnNcACkmrJN7gIys5riAEg15Zoh1+MAMLOa4QBINeeafA7AzGqKAyDVUu8WgJnVlrICQNIKSeskrZd0Y4n5p0n6qaQ+SR8rKD9V0mMFQ7ekG9J5N0naVDDv0rF7W5VryjWhercAzKx2jPqbwJIywJeBi4AuYI2k1RHxTMFiO4HrgSsK142IdcBZBdvZBHyvYJG/jojPHdM7GCPNuWbqGtwCMLPaUU4LYDmwPiI2RMRB4E5gZeECEbEtItYA/SNs50Lg+Yh48ahrO46ask3I5wDMrIaUEwAdwEsF011pWaWuBL5RVHadpCck3S5p9lFsc8w055pRvVsAZlY7ygkAlSiLSnYiqR64HPh2QfHNwMkkXURbgM8Ps+41kjoldW7fvr2S3VakKef7AMystpQTAF3AooLphcDmCvdzCfBoRGzNF0TE1ogYjIgh4CskXU2vEhG3RsSyiFjW3t5e4W7L15RtIrJuAZhZ7SgnANYASyUtSb/JXwmsrnA/V1HU/SNpQcHkO4GnKtzmmGrONTOUcQvAzGrHqFcBRcSApOuAu4EMcHtEPC3p2nT+LZKOBzqBNmAovdTz9IjoltRMcgXRh4s2/RlJZ5F0J20sMX9CNWWbGMq4BWBmtWPUAACIiLuAu4rKbikYf5mka6jUuj3A3BLl76+opuOsOdcMdYP09PUDuWpXx8xs3PlO4FRTrgmAA/09Va6JmdnEcACk8j8Kc2DgQJVrYmY2MRwAqaZs2gIYcAvAzGqDAyCVbwH0DboFYGa1wQGQyp8D6Au3AMysNjgAUvkWwMEhtwDMrDY4AFL5cwAHo4eo6EEXZmZTkwMglW8BkD3AwYPVrYuZ2URwAKTy5wD8q2BmViscAKl8F5B/F9jMaoUDIHWoCyjXwwGfBzazGuAASB3qAsoeYP/+6tbFzGwiOABSjdnGZCTXw7591a2LmdlEcACk6lRHfV0j5A44AMysJjgACjRlmiHX4y4gM6sJDoACjdnkd4HdAjCzWuAAKNCccwvAzGqHA6BAc32TzwGYWc1wABSY0dDsq4DMrGaUFQCSVkhaJ2m9pBtLzD9N0k8l9Un6WNG8jZKelPSYpM6C8jmS7pX0XPo6+9jfzrFpzjWhet8HYGa1YdQAkJQBvgxcApwOXCXp9KLFdgLXA58bZjO/GhFnRcSygrIbgfsiYilwXzpdVc25Zurq3QVkZrWhnBbAcmB9RGyIiIPAncDKwgUiYltErAH6K9j3SmBVOr4KuKKCdcdFU64J1bsLyMxqQzkB0AG8VDDdlZaVK4B7JD0i6ZqC8uMiYgtA+jq/1MqSrpHUKalz+/btFey2ck3ZJpRzF5CZ1YZyAkAlyir5yZS3RMQ5JF1IH5F0QQXrEhG3RsSyiFjW3t5eyaoVa841Ez4JbGY1opwA6AIWFUwvBDaXu4OI2Jy+bgO+R9KlBLBV0gKA9HVbudscL03ZJiLjcwBmVhvKCYA1wFJJSyTVA1cCq8vZuKQWSa35ceDXgafS2auBq9Pxq4HvV1Lx8dCca2Yo4xvBzKw2ZEdbICIGJF0H3A1kgNsj4mlJ16bzb5F0PNAJtAFDkm4guWJoHvA9Sfl9fT0i/jXd9KeAb0n6EPBL4N1j+9Yq15RrIjTI3v39QK7a1TEzG1ejBgBARNwF3FVUdkvB+MskXUPFuoE3DLPNHcCFZdd0AuR/FGZvXw8ws7qVMTMbZ74TuED+ZyH39/knwcxs+nMAFGipbwFgf/8+opLrnMzMpiAHQIHW+lYAIrfXPwxvZtOeA6BAW0NbMtKw11cCmdm05wAocDgAun0vgJlNew6AAq0NSReQA8DMaoEDoMChFkC9u4DMbPpzABTInwR2C8DMaoEDoEBLfQtC0LDXAWBm054DoECd6mjJtUJDt7uAzGzacwAUmZEGgFsAZjbdOQCKzGxsg3p3AZnZ9OcAKNLW6C4gM6sNDoAiMxvbUKNbAGY2/TkAirQ1tFHX5HMAZjb9OQCKtNa7C8jMaoMDoEhbQxvhk8BmVgMcAEVa61sZynWzd59/EMDMpreyAkDSCknrJK2XdGOJ+adJ+qmkPkkfKyhfJOmHktZKelrSHxbMu0nSJkmPpcOlY/OWjk1bQxtoiL0H/KtgZja9jfqbwJIywJeBi4AuYI2k1RHxTMFiO4HrgSuKVh8A/jgiHpXUCjwi6d6Cdf86Ij53zO9iDOUfCLentxtorm5lzMzGUTktgOXA+ojYEBEHgTuBlYULRMS2iFgD9BeVb4mIR9PxvcBaoGNMaj5O8o+ETgLAzGz6KicAOoCXCqa7OIoPcUknAmcDDxcUXyfpCUm3S5o9zHrXSOqU1Ll9+/ZKd1uxQy2AA3vHfV9mZtVUTgCoRFlFZ0glzQC+A9wQEfmv1jcDJwNnAVuAz5daNyJujYhlEbGsvb29kt0elfwjoff1dzM4OO67MzOrmnICoAtYVDC9ENhc7g4k5Ug+/L8WEd/Nl0fE1ogYjIgh4CskXU1VV/izkLt3V7cuZmbjqZwAWAMslbREUj1wJbC6nI1LEnAbsDYivlA0b0HB5DuBp8qr8vgq/GH4nTurWxczs/E06lVAETEg6TrgbiAD3B4RT0u6Np1/i6TjgU6gDRiSdANwOnAm8H7gSUmPpZv8s4i4C/iMpLNIupM2Ah8e27d2dAp/F3jXrurWxcxsPI0aAADpB/ZdRWW3FIy/TNI1VOxBSp9DICLeX341J07h7wI7AMxsOvOdwEWask3UqQ4aut0FZGbTmgOgiCRac23uAjKzac8BUEJrY6tPApvZtOcAKGFmQxuZZrcAzGx6cwCU0NbQRrbFLQAzm94cACW0NrRS1+QWgJlNbw6AEmY1zoLGXQ4AM5vWHAAlzGuax0D9K+4CMrNpzQFQQntLO/3ZXezcPVDtqpiZjRsHQAnzmucBsPOAmwBmNn05AErIB0Bv3Xb6+qpcGTOzceIAKKG9Of3dgeZXfCLYzKYtB0AJ+RYALdt9ItjMpi0HQAntLW4BmNn05wAoYW7T3GSk2S0AM5u+HAAlNGQbmJFrcwvAzKY1B8Aw5jXNg5btDgAzm7YcAMOY39oOza+wY0e1a2JmNj7KCgBJKyStk7Re0o0l5p8m6aeS+iR9rJx1Jc2RdK+k59LX2cf+dsZOe/M8Mq2vsH17tWtiZjY+Rg0ASRngy8AlJD/0fpWk04sW2wlcD3yugnVvBO6LiKXAfen0pDGveR6asZ2XX652TczMxkc5LYDlwPqI2BARB4E7gZWFC0TEtohYA/RXsO5KYFU6vgq44ijfw7hob25nqOEVXt4a1a6Kmdm4KCcAOoCXCqa70rJyjLTucRGxBSB9nV9qA5KukdQpqXP7BPbHzGuex1Cmly2v7J+wfZqZTaRyAkAlysr9Wnws6yYLR9waEcsiYll7e3slqx6T/M1gW/e+QrgRYGbTUDkB0AUsKpheCGwuc/sjrbtV0gKA9HVbmducEIUPhNu3r8qVMTMbB+UEwBpgqaQlkuqBK4HVZW5/pHVXA1en41cD3y+/2uOv8IFwPhFsZtNRdrQFImJA0nXA3UAGuD0inpZ0bTr/FknHA51AGzAk6Qbg9IjoLrVuuulPAd+S9CHgl8C7x/rNHYvCB8Jt3QpLl1a3PmZmY23UAACIiLuAu4rKbikYf5mke6esddPyHcCFlVR2IhU+EM4tADObjnwn8DBmNswkV5eDlm0OADOblhwAw5DEghkLoHUzW7dWuzZmZmPPATCChTMXUj+vyy0AM5uWHAAj6GjtQG2bHABmNi05AEawsG0hA81dfhyEmU1LDoARdLR2MJjpYcvOPdWuipnZmHMAjGBhW3Jl69beLj8OwsymHQfACDrakufWDTRuYvfuKlfGzGyMOQBGkG8B0OYrgcxs+nEAjOCE1hOSkbZNdHVVty5mZmPNATCC+kw98xrnQ1sXL7xQ7dqYmY0tB8AoFs3qQDM3sWFDtWtiZja2HACjWNi2kNxctwDMbPpxAIyio7WDmOEWgJlNPw6AUSxsW0h/bgfPv3ig2lUxMxtTDoBR5O8F2DWwmT2+IdjMphEHwCgWz1ycjMza6PMAZjatOABGccrcU5KRuc/6PICZTStlBYCkFZLWSVov6cYS8yXpb9P5T0g6Jy0/VdJjBUN3+nvBSLpJ0qaCeZeO7VsbGx2tHbTkWmDeOgeAmU0ro/4msKQM8GXgIqALWCNpdUQ8U7DYJcDSdDgPuBk4LyLWAWcVbGcT8L2C9f46Ij43Fm9kvEjilLmn8OTxDgAzm17KaQEsB9ZHxIaIOAjcCawsWmYlcEckHgJmSVpQtMyFwPMR8eIx13qCnTrvVOraf+FzAGY2rZQTAB3ASwXTXWlZpctcCXyjqOy6tMvodkmzS+1c0jWSOiV1bt++vYzqjr1T557KweYXeW6jLwU1s+mjnABQibLip+OPuIykeuBy4NsF828GTibpItoCfL7UziPi1ohYFhHL2tvby6ju2Dtt3mmgYOOe9fT1VaUKZmZjrpwA6AIWFUwvBDZXuMwlwKMRsTVfEBFbI2IwIoaAr5B0NU1Kp849FYDB2et45plRFjYzmyLKCYA1wFJJS9Jv8lcCq4uWWQ18IL0a6E3AnojYUjD/Koq6f4rOEbwTeKri2k+Qw5eCruOJJ6pbFzOzsTLqVUARMSDpOuBuIAPcHhFPS7o2nX8LcBdwKbAe6AF+N7++pGaSK4g+XLTpz0g6i6SraGOJ+ZNGS30LC1sXsvm4dTz+eLVrY2Y2NkYNAICIuIvkQ76w7JaC8QA+Msy6PcDcEuXvr6imVXbqvFPZ1eEAMLPpw3cCl+m0eadxcOZaHnt8yD8Qb2bTggOgTGcffzb9dXvZGc+zufgUuJnZFOQAKNO5J5ybjJzwiLuBzGxacACU6Yz2M2jINMCCR3wlkJlNCw6AMuUyOc487kwaT3qEzs5q18bM7Ng5ACpw7oJzGZz/KPc/ED4RbGZTngOgAueecC79mT28Mvg8a9dWuzZmZsfGAVCBcxack4wseIT7769uXczMjpUDoAKvn/966jP1zDil0wFgZlOeA6AC9Zl63njCG6k/9Yfcfz8+D2BmU5oDoEIrXruCnY2P8PLebTz3XLVrY2Z29BwAFbrktZckIyffzQ9+UN26mJkdCwdAhc5ecDbzW+Yzc9m/8k//VO3amJkdPQdAhepUx4rXrqB/8d08+JNBPxfIzKYsB8BRWHHyCnrYAQs6+e53q10bM7Oj4wA4Citeu4L6TD1z3/ZNvv3t0Zc3M5uMHABHYXbTbN6x9B30nfINHnhwgA0bql0jM7PKOQCO0vvPfD/7eJm6197Hl75U7dqYmVWurACQtELSOknrJd1YYr4k/W06/wlJ5xTM2yjpSUmPSeosKJ8j6V5Jz6Wvs8fmLU2MS5deyqzGWSz+jX/kttugu7vaNTIzq8yoASApA3wZuAQ4HbhK0ulFi10CLE2Ha4Cbi+b/akScFRHLCspuBO6LiKXAfen0lNGQbeA9p7+Hl2d/l739u7jttmrXyMysMuW0AJYD6yNiQ0QcBO4EVhYtsxK4IxIPAbMkLRhluyuBVen4KuCKCuo9KXxk+UfoHezhNb/9JT79abcCzGxqKScAOoCXCqa70rJylwngHkmPSLqmYJnjImILQPo6v9TOJV0jqVNS5/bt28uo7sQ587gzufzUy9l9yhfZunsv/+N/VLtGZmblKycAVKKs+DFoIy3zlog4h6Sb6COSLqigfkTErRGxLCKWtbe3V7LqhPj4+R9nT/9O3vj7N/PFL8IvflHtGpmZlaecAOgCFhVMLwSK738ddpmIyL9uA75H0qUEsDXfTZS+bqu08pPB8o7lXHzyxTw7/3/SNG87f/iHfkqomU0N5QTAGmCppCWS6oErgdVFy6wGPpBeDfQmYE9EbJHUIqkVQFIL8OvAUwXrXJ2OXw18/xjfS9V84eIvsH9gH6+//uPccw+sLj46ZmaT0KgBEBEDwHXA3cBa4FsR8bSkayVdmy52F7ABWA98BfgvaflxwIOSHgd+BvxLRPxrOu9TwEWSngMuSqenpNPbT+cPlv8BDx38B5a89Wf80R/B/v3VrpWZ2cgUU6i/YtmyZdHZ2Tn6glWwp3cPZ/yvM9BAM11//ijv/+0ZrFoFKnV2xMxsAkl6pOgyfMB3Ao+ZmY0z+dpvfo3Nvc/zhj+7jq9+Fb7ylWrXysxseA6AMfS2E9/GJ87/BI+zitd94Mt85CM+H2Bmk5cDYIz9xdv+gstOuYx1J1/PSRf/gHe/G+65p9q1MjN7NQfAGMvUZfj6u77OmcedyUu/8i463nY3l10G35+y1ziZ2XTlABgHM+pncPf77uaUuafQdf5lLHzHHbzrXfBXfwUDA9WunZlZwgEwTua3zOeHV/+QNy96MxvecDULr38vf/5Xe7jgAli/vtq1MzNzAIyr2U2zue8D9/GXv/qXdM36JvP+/A08uedHnHUW/M3fwMGD1a6hmdUyB8A4y9Rl+MQFn+DBDz5IW2uGfe+5gFlXf4g/+sRWzjgDvvMdPzrCzKrDATBB3rTwTTz24cf4r2/+r2w9/g4a/nQJO5ffwG9ds55zz4U/+RP4yU+qXUszqyUOgAnU2tDKZy76DM/8l2e46szfZs+pfwfXL2X9+W/nC/d9lbe8vYcrroAf/9itAjMbf34URBVt3ruZVY+t4raf38bzu54nRxOx/iIG1r+dDs7jN3/lbFa+o4nzz4f6+mrX1symquEeBeEAmASGYogHXnyA7zzzHf553f/lxe6NyYzBLGw9k+y28/gPc5Zz6X84j8Utp7J4UR0XX+znDJlZeRwAU8iWvVv42aaf8eDGh7nnmYdZu2cN/XV7k5m9M2HTG+nQcn7zLWfyttcv5cKzljKrubW6lTazScsBMIUNxRBrt/+Cf33yYX6+/WEe2PAwL/U9CXWDh5bJ9rUzq66DBTM6OHn+CZyxuIMT53Rw4qwTWTJrCYtnLiaXyVXxXZhZtTgAppk9PT3c07me+596jp+/+Bwbdr3AK32bGGjaDK2bYMaRP7Am6mjOzqC1oZkT2k5gUduiZJi5iIVtC6nP1JOryzGnaQ5zmuYwt3kuc5rmUJ/xyQezqc4BUAMi4IUX4LHHoPPnB3l8w2Z2DG7kxT0v8HLvRmjopq5xH43zN1E36yX6mn5Jf133iNtsybUcCoPZjbPJZXJklOE1M1/DopmLmFE/g5ZcS/Ja30JLroWW+hZa61tpa2ijtaGVGfUzqJMvODOrFgdAjdu2DR58EB5+OAmJF19Mhq27u6F1M9T1Q+YgNO2i9bgdqGknxy/ZyYKTd0DTTg5mdzCQ3UVddoD+oX427t7IzgM7y95/a30rrQ1pKKTh0NbQRkO2gb6BPhqzjUnLo2kus5tm05BpoDHbSEO2gYZMw7CvpZbJ1mWRz5CbHeIAsJL27UuCoK4Odu2Cf/932LQJenvhgQdgw4Yjl6+vhxNPhJNOgsUn9dKxZD9zjtvPjDn7aZq5j8bW/dQ17qenfx/dfd1093Wz9+De0uN9e+kd6KUh20DvQC87enawu3c3wbH9TQodEQqN2UZydTmydVlymeQ1W5eltb71UJfXzIaZh5YtDJNcJneoe6wx21jWUJ+pdwDZpDJcAGTLXHkF8EUgA/xDRHyqaL7S+ZcCPcDvRMSjkhYBdwDHA0PArRHxxXSdm4D/DGxPN/NnEXHXUbw3OwYzZsAZZxyefvObD49HwNatsGNHEgovvJAEQn546KFGdu9uBOYesc1sFtrbYf58aG5O9vGWt8BFy+G4hXDcccm8XIlz0oNDg3T3ddM32EffQN8Rr70Dva8qK3fewFDSchkYGjg07O3by9pX1rKjZwd7+vbQN9B3zOEDhwNI6ND2hMhlcuTqciO+tta3srBtIS25FgZjkMGhQSQdakG11rfSkG04FGIZZQ6NZ+uyZOoy1KmOOtWR0eHxbF32UEA1ZBsOzZOUvKJDyzblmsjWZenp76Gnv4cD/QfI1GWoz9RTn6mnIdNwaLwx2+gW1xQ2agtAUgZ4luSH27uANcBVEfFMwTKXAn9AEgDnAV+MiPMkLQAWpGHQCjwCXBERz6QBsC8iPlduZd0CmHx27YLNm5Og2LYtec0P27ZBXx+88kpyXqL4T23OnCQMyhkaGsb/vUQEA0MDR4RH/2A/BwcP0j/UT99AEibFw4GBAyXL84QYjEH6B/vpH+o//Fo4nr5293XT1d3Fgf4DyYd4XYahGGLfwX309PeM/0E4CvnAK2wBDQ4NHgpaSJ6JlQ+r/HimLnMoxErNzwdSqZAarixXl6M510xGmUP1C4KhGDr8Goen61RHfV0abNkk2OpUx8DQwBHvYWBogMEYLDkOHBHC+dbmcEP+76zwC8lQDB1xHEq9vu/M9/HaOcZpbOsAAAvJSURBVK89un+jY2gBLAfWR8SGdEN3AiuBZwqWWQncEUmaPCRplqQFEbEF2AIQEXslrQU6ita1KWz27GQobEWUsmsXrF17ZEAUDo88krzu3Vt6/Zkzyw+L5uajey9S+i09k2NG/Yyj28g4GhwaZN/Bfa9qyRQOQzHEUAwxODR4eDz9sCoMsMEYPOKDMP/BODA0QO9AL/1D/bTkWmjONdOYbWQohjg4ePDQUKqlld9+32DfES0U4FBrJl+XV02n4/kP3Xz98u+p+MN7uLL+oX56+nsYiqEjjl1hWEhCCEnJOmnI54fBGHzVh3ZhSysfXMXvb2BogP7B4f9t8h/6+RZZYVhIOuIYFL8Gwa8s/JWjDoDhlBMAHcBLBdNdJN/yR1umg/TDH0DSicDZwMMFy10n6QNAJ/DHEbGr3Irb1DJ79pHdS8M5cGD4kMgPTz4J//ZvsHt36W3MmFFeUNTVwdAQLF48NR61kanLMLNxZrWrYVUwXudqywmAUp17xbUZcRlJM4DvADdERP66w5uBv0yX+0vg88AHX7Vz6RrgGoDFixeXUV2bypqakpPMJ544+rJ9fa/udioe1q1LTmbv2DH8drLZJATyrZnZs2HuXJg379Wv+fHWVj+KwybOeJ1jKScAuoBFBdMLgc3lLiMpR/Lh/7WI+G5+gYjYmh+X9BXg/5baeUTcCtwKyTmAMuprNaKhARYtSobR9PfD9u1Hnp+ISIb165MT3Lt2Ja2Kl16CnTuT0BgaKr29XG7kgCgenzsX2tpGD42hIbjjjiQA3/72So+IWWXKCYA1wFJJS4BNwJXAfypaZjVJd86dJN1DeyJiS3p10G3A2oj4QuEKBecIAN4JPHUM78NsRLkcnHBCMpRraCgJhB07khPZr7xSenzHDnjmmWR8504YHCy9vWz21aEwN72A6uBBWLgwaa386EeQycCXvgQf/ODYngB/+WV497vh938f/lPx/2KrOWXdB5Be5fM3JJeB3h4Rn5R0LUBE3JJ+0P8dsILkMtDfjYhOSW8FfgQ8SXIZKKSXe0r6KnAWSRfQRuDDBYFQkq8CssluaAj27Hl1QIwUIHV1SThs2ZJ0LX32s/B//g/8y78k22xshFmzkmHmzNLj+W6rwmH27OSEeL7VcfAg/NqvJb830dSUnHh/3euqd6xs4vhGMLNJbmAgCZD6+mT8G99IuqN2706GPXsOj+end+0a+belM5mk66mtLZl+8cXk96g/+cnkMtzf+q0kSPJDPlgKp5uakhDp6YH770/q9/a3J9s+WhE+hzKRHABm01Rv7+FzFoXD7t3Q3Z0Me/Ykw8UXJ90/994LV1+dnA8Z7jxHXjabtEL6+pJzKQAdHcmlvyO1TOrrk4BqaYGTT05aJU8+CR/9aBIe//iPboFMlGO6E9jMJq/GxsrPb1x0UXIDX0TyOJB8QORbGYXTe/YkIdPQkHQhdXcnrZOurqRFkV+nt3f0/QK85jWwfz+cey688Y1Jt1e+lVI4PtL0jBlJvV94AZYsSULHKucAMKthUvKh2tqanIQu17ve9eqy3t4jQ+TgweQ8RHd38uiQnTuT7qT3vjdZ5uMfT8q3bIFnnz3cWjlwoLL3UFcHb3hDcmI9/17mzElaHA0NSQsml0taJE1NcPzxyaNI6uqSsny45Lu6aom7gMxsUhkYSO4IzwdC4Xh+es+e5AN7yRJ4+unkKbd79iTz9u5Nwmbfvsr2m80ebl00Nyfbb2lJgmX27CRE8kGSyyXzjzsuWX5gIFl2zpxkmDkzmW5pSbY13PmSvj64+ebkyrHrry/9fKyx4C4gM5sSstnDN+Qdi97e5JzFwEDy2t+fdD1t2ZLcEwLJB3BhuOQD5sCB5KT3/v3w/PNJoBRup78/Wbfc788NDUkQNDcf2a21fj1s3Jgss2oVXHZZskxheDQ1JesvX548ZHEsOQDMbFpqbEyGYqecMjbbHxhIbijs6UlCa9++5KT3zp1JF1hPz+EQyb/u358slw+bJUvg1luTwPnoR+HTnx7+PpIf/ABWrBibuuc5AMzMjkI2W9mJ99FcfnnSosi3VPJDb2/S2hir4CrkADAzmySk5BxDff2xd4GVwz/UamZWoxwAZmY1ygFgZlajHABmZjXKAWBmVqMcAGZmNcoBYGZWoxwAZmY1ygFgZlajHABmZjXKAWBmVqPKCgBJKyStk7Re0o0l5kvS36bzn5B0zmjrSpoj6V5Jz6WvE/DkCzMzyxs1ACRlgC8DlwCnA1dJOr1osUuApelwDXBzGeveCNwXEUuB+9JpMzObIOW0AJYD6yNiQ0QcBO4EVhYtsxK4IxIPAbMkLRhl3ZXAqnR8FXDFMb4XMzOrQDmPg+4AXiqY7gLOK2OZjlHWPS4itgBExBZJ80vtXNI1JK0KgH2S1pVR51LmAa8c5boTbarUdarUE6ZOXadKPWHq1HWq1BPGr66vKVVYTgCU+pnk4h9CG26ZctYdUUTcCtxayTqlSOos9ZuYk9FUqetUqSdMnbpOlXrC1KnrVKknTHxdy+kC6gIWFUwvBDaXucxI625Nu4lIX7eVX20zMztW5QTAGmCppCWS6oErgdVFy6wGPpBeDfQmYE/avTPSuquBq9Pxq4HvH+N7MTOzCozaBRQRA5KuA+4GMsDtEfG0pGvT+bcAdwGXAuuBHuB3R1o33fSngG9J+hDwS+DdY/rOXu2Yu5Em0FSp61SpJ0yduk6VesLUqetUqSdMcF0VUVGXvJmZTRO+E9jMrEY5AMzMalRNBMBoj7KoFkmLJP1Q0lpJT0v6w7T8JkmbJD2WDpdWu64AkjZKejKtU2daNqke6SHp1ILj9pikbkk3TJZjKul2SdskPVVQNuwxlPTf0r/bdZIurnI9PyvpF+njXr4naVZafqKkAwXH9paJqucIdR3233uSHdNvFtRxo6TH0vKJOaYRMa0HkpPPzwMnAfXA48Dp1a5XWrcFwDnpeCvwLMkjM24CPlbt+pWo70ZgXlHZZ4Ab0/EbgU9Xu55F//Yvk9wEMymOKXABcA7w1GjHMP1beBxoAJakf8eZKtbz14FsOv7pgnqeWLjcJDmmJf+9J9sxLZr/eeAvJvKY1kILoJxHWVRFRGyJiEfT8b3AWpK7p6eSyfxIjwuB5yPixWpXJC8iHgB2FhUPdwxXAndGRF9EvEByld3yatUzIu6JiIF08iGS+3qqbphjOpxJdUzzJAl4D/CNiahLXi0EwHCPqZhUJJ0InA08nBZdlza1b692t0qBAO6R9Ej6iA4oeqQHUPKRHlVyJUf+h5qMxxSGP4aT+W/3g8APCqaXSPq5pPslnV+tShUp9e89WY/p+cDWiHiuoGzcj2ktBMAxP45ivEmaAXwHuCEiukmepnoycBawhaRpOBm8JSLOIXm660ckXVDtCg0nvfHwcuDbadFkPaYjmZR/u5I+DgwAX0uLtgCLI+Js4KPA1yW1Vat+qeH+vSflMQWu4sgvKxNyTGshAMp5lEXVSMqRfPh/LSK+CxARWyNiMCKGgK8wQU3U0UTE5vR1G/A9knpN1kd6XAI8GhFbYfIe09Rwx3DS/e1Kuhr4DeC9kXZWp90pO9LxR0j61U+pXi1H/PeejMc0C/wm8M182UQd01oIgHIeZVEVab/fbcDaiPhCQfmCgsXeCTxVvO5Ek9QiqTU/TnJC8Ckm7yM9jvhGNRmPaYHhjuFq4EpJDZKWkPzexs+qUD8guZoO+FPg8ojoKShvV/LbH0g6iaSeG6pTy0N1Gu7fe1Id09R/BH4REV35ggk7phNx9rvaA8ljKp4lSdGPV7s+BfV6K0nz8wngsXS4FPgq8GRavhpYMAnqehLJ1ROPA0/njyMwl+QHfZ5LX+dMgro2AzuAmQVlk+KYkoTSFqCf5Nvoh0Y6hsDH07/bdcAlVa7nepL+8/zf6i3psu9K/yYeBx4FLpsEx3TYf+/JdEzT8v8NXFu07IQcUz8KwsysRtVCF5CZmZXgADAzq1EOADOzGuUAMDOrUQ4AM7Ma5QAwM6tRDgAzsxr1/wF/h7Wjo9FGjAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_type_loss', 'Wine Type Loss', ylim=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uYV9AOAMwI9p"
   },
   "source": [
    "### Plots for Confusion Matrix\n",
    "\n",
    "Plot the confusion matrices for wine type. You can see that the model performs well for prediction of wine type from the confusion matrix and the loss metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C3hvTYxIaf3n"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAEQCAYAAAAkgGgxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3debxVdb3/8debQZRJmUUEnNAuaqmZ2aCRZqJZWjdvOBSVXaufdq0sU+/95VD0s2s53MqyweJmhuSQpCYSZmqpqGgqk5AkIMgoKqgI53x+f6zvkS2evc9ecDb77MX7yWM99l7f/V3f9d3D+fAd1qCIwMysiDrVuwJmZrXiAGdmheUAZ2aF5QBnZoXlAGdmheUAZ2aF5QC3CUk7SPqDpBck/W4LyjlF0p3tWbd6kXSYpDk1KDf3Zy3pbkmfa++6bLKPT0u6r4bl/1HS2JL1b0taIek5ScMkrZHUuVb735Z0qXcFNpekk4GvAm8BXgIeA8ZFxJb+MD8ODAL6RcSGzS0kIn4D/GYL61JzkgIYERHzyuWJiHuBfWqw+4qftaQLgb0i4tQa7LtuIuKYlueShgJnA8MjYllK7lmXihVQQ7bgJH0VuAL4DtkfyDDgKuD4dih+OPDUlgS3IpFUy/8E/Vlnn8HKkuC22Wr8XTWmiGioBdgRWAOcWCFPN7IAuDgtVwDd0mujgEVk/2suA5YAn0mvXQS8BqxP+zgNuBC4tqTs3YAAuqT1TwNPk7Ui5wOnlKTfV7Ldu4GHgBfS47tLXrsb+Bbw11TOnUD/Mu+tpf7nlNT/BOBY4ClgFXB+Sf5DgPuB1SnvD4Ht0mv3pPeyNr3fT5SU/w3gOeDXLWlpmz3TPg5K67sAK4BRZer7L+n9rQZmAB8p91lvst3oTV7/ezWfFXAo8Le0v7+Xq1fKOxS4CVgOrAR+WOa7uxJYCLwIPAIctsnn+3B6bSlwWUrfHrg2lbs6feeDSt7D54APAK8Azek9/oo3/752BH6RvrtngW8DnUvq+Vfg8vSdfLvef58dbal7BXJXOPvhb2j5AZTJczHwADAQGJB+8N9Kr41K218MdCULDC8DfdLrF/LGgLbp+us/QKBH+mHvk14bDOxb8uO7Lz3vCzwPfDJtd1Ja75devxv4B7A3sENav6TMe2up/zdT/f89/YFeB/QC9gVeBfZI+d9O9kffJdV9FvDlkvKCrBu4afnfJfuPYgdKAlzK8++pnO7AZOB7ZeraFZgHnA9sBxxBFpT2ae2zbWX7N71e6bMChpAFlGPJeidHpfUBrZTdmSwAXp6+x+2B92763aX1U4F+6TM8myzwb59eux/4ZHreEzg0Pf888If0GXVO30PvkvfwuZLPu/Sz3Y03BrjfA1enOg4EpgGfL6nnBuBLqW471Pvvs6MtjdhF7QesiMrdmlOAiyNiWUQsJ2stfLLk9fXp9fURcTvZ/56bO8bUDOwnaYeIWBIRM1rJ8yFgbkT8OiI2RMRvgdnAh0vy/DIinoqIV4CJwAEV9rmebLxxPTAB6A9cGREvpf3PAN4KEBGPRMQDab//JPtjeV8V7+mCiFiX6vMGEfEzYC7wIFlQ/88y5RxK9kd/SUS8FhF3AbeSBfgtUe6zOhW4PSJuj4jmiJhC1ro6tpUyDiFrfX49ItZGxKtRZvw2Iq6NiJXpM/w+WeBv+b2sB/aS1D8i1kTEAyXp/cj+82hK38OLed6kpEHAMWT/Ia2NrBt7OTCmJNviiPhBqtubvqttXSMGuJVA/zbGG3YBnilZfyalvV7GJgHyZTZjYDci1pJ1674ALJF0m6S3VFGfljoNKVl/Lkd9VkZEU3re8qNeWvL6Ky3bS9pb0q1phu5FsnHL/hXKBlgeEa+2kednwH7ADyJiXZk8uwALI6K5JG3T9705yn1Ww4ETJa1uWYD3kgXhTQ0FnmnjP0oAJJ0taVaa7V1N1m1s+QxPI2tNzpb0kKTjUvqvyVq3EyQtlvTfkrrmfJ/DyVrBS0rez9VkLbkWC3OWuU1pxAB3P1kX7IQKeRaT/ThaDEtpm2MtWTejxc6lL0bE5Ig4iuyPaDbZH35b9Wmp07ObWac8fkxWrxER0Zusu6g2tql4iRlJPcnGNX8BXCipb5msi4Ghkkp/Z3ned95L3SwEfh0RO5UsPSLikjJ5h7U1MC/pMLLxyH8jG8bYiWwcVQARMTciTiILOt8FbpDUI/UOLoqIkWTjr8cBn9qM97OObIyx5f30joh9S/L4ckAVNFyAi4gXyMaffiTpBEndJXWVdIyk/07Zfgv8l6QBkvqn/Ndu5i4fAw5PxyftCJzX8oKkQZI+IqkH2Q9xDdDUShm3A3tLOllSF0mfAEaSdddqrRfZOOGa1Lr84iavLwX2yFnmlcAjEfE54DbgJ2XyPUj2H8Q56TsaRdYtn1DlfpYCu20SICu5FviwpKMldZa0vaRRknZtJe80soH7SyT1SHnf00q+XmTjXMuBLpK+CfRueVHSqZIGpFbq6pTcJOn9kvZPx7O9SNZlbe23UVZELCGbRPm+pN6SOknaU1JbQwyWNFyAA4iIy8iOgfsvsh/eQuBMsgFZyGaaHgYeB54Apqe0zdnXFOD6VNYjvDEodSIbdF5MNov1PuD/tFLGSrL/wc8m62KfAxwXESs2p045fQ04mWxw/2dk76XUhcD41AX6t7YKk3Q82UTPF1LSV4GDJJ2yad6IeA34CNk40gqyQ3k+FRGzq6x7y8G/KyVNbytzRCwkO1TofDb+Lr5OK7/z1MX/MLAXsIBs5vgTrRQ7Gfgj2Qz1M2S9h9Ju4WhghqQ1ZIF/TOre7wzcQBbcZgF/YfP+k/0U2QTNTLKJqRtovcttrVCEW7i1Imk02Y++M/DzMl0l60AkXUP2n9GyiNiv3vWxLdOQLbhGkLomPyJrvYwETpI0sr61sir8iqxVZgXgAFc7hwDzIuLp1FWbQPucaWE1FBH3kA03WAE4wNXOEN44VrOILT88wsxycICrndYOxfCAp9lW5ABXO4vIDiZtsSubfyyemW0GB7jaeQgYIWl3SduRnV4zqc51MtumOMDVSDoF6Eyy46hmARPLnKdqHYik35KdLbOPpEWSTqt3nWzz+Tg4Mysst+DMrLAc4MyssBzgzKywHODMrLAc4LYCSafXuw6Wj7+zYnCA2zr8x9J4/J0VgAOcmRVWhzoOrn/fzrHb0LyXre/4lq9sYkC/Yt6o/KnHu7edqQGtZx1d6VbvarS7V1nLa7GurUvWV3T0+3vEylXVXZz4kcfXTY6Iul1+qkPdKHa3oV2ZNnlo2xmtwzh6l0o3/7KO5sGYusVlrFzVxLTJw6rK23nw3LZucFRTHSrAmVnHF0AzzW3m6wgc4MwslyBYH7nun1M3nmQws9yaq/xXiaR9JD1Wsrwo6cuS+kqaImlueuxTss15kuZJmiPp6Lbq6QBnZrkEQVNUt1QsJ2JORBwQEQcAbye7iffNwLnA1IgYAUxN66R7mowB9iW7b8ZV6d4nZTnAmVluzURVSw5HAv+IiGfI7l0yPqWPZ+NN3o8HJkTEuoiYD8wju/dJWR6DM7NcAmhq/6vvjyG7YTvAoHTTayJiiaSBKX0I8EDJNm3e58QtODPLLUcLrr+kh0uWN50hkq54/RE23ui7nNz3OXELzsxyCWB99ScIrIiIg9vIcwwwPSKWpvWlkgan1ttgYFlKz32fE7fgzCyXIGiqcqnSSWzsnkJ275Kx6flY4JaS9DGSuknaHRgBTKtUsFtwZpZPQFM7DcFJ6g4cBXy+JPkSYGK6H8YC4ESAiJghaSIwE9gAnBFR+YA8BzgzyyU7k6Gdyop4Gei3SdpKslnV1vKPA8ZVW74DnJnlJJpaHe/veBzgzCyXbJLBAc7MCig7Ds4BzswKqtktODMrIrfgzKywAtHUIIfQOsCZWW7uoppZIQXitWiMe4w4wJlZLtmBvu6imllBeZLBzAopQjSFW3BmVlDNbsGZWRFlkwyNEToao5Zm1mF4ksHMCq3Jx8GZWRH5TAYzK7Rmz6KaWRFlJ9s7wJlZAQVivU/VMrMiisAH+ppZUalhDvRtjDBsZh1GkLXgqlnaImknSTdImi1plqR3SeoraYqkuemxT0n+8yTNkzRH0tFtle8AZ2a5NdGpqqUKVwJ3RMRbgLcBs4BzgakRMQKYmtaRNBIYA+wLjAauklRxMNABzsxyCURzVLdUIqk3cDjwC4CIeC0iVgPHA+NTtvHACen58cCEiFgXEfOBecAhlfbhMTgzyyW7bWC7hI49gOXALyW9DXgEOAsYFBFLACJiiaSBKf8Q4IGS7ReltLLcgjOznLIbP1ezAP0lPVyynF5SUBfgIODHEXEgsJbUHS274zeLSjV1C87McglyncmwIiIOLvPaImBRRDyY1m8gC3BLJQ1OrbfBwLKS/ENLtt8VWFxp527BmVluOVpwZUXEc8BCSfukpCOBmcAkYGxKGwvckp5PAsZI6iZpd2AEMK3SPtyCM7NcItSe56J+CfiNpO2Ap4HPkDW8Jko6DVgAnJjtN2ZImkgWBDcAZ0REU6XCHeDMLJdskqF9TtWKiMeA1rqwR5bJPw4YV235DnBmlpPvyWBmBZVNMjTGqVoOcGaWmy+XZGaF1HImQyNwgNtCc+a9xklfeO719aefWc9FX+/Hs89t4NY717LddmKP4V255oqB7LTjxoHZBYvWs9/7FnDB1/py9hf7tFa01cHZv/gi7/zQ21m97AVOf+vZ9a5Oh9UoN51pjFp2YPvstR3T/zSM6X8axkOTh9J9h06ccEwPPnB4dx6/exiP3TWMvffsyiU/eP4N2331ghWMPqJ7nWpt5dz5q7s5/5iqJ+m2SRGwvrlTVUu9uQXXjqbe+wp77taV4UOzpcU7D9qeG29d8/r67/+4hj2Gd6VH98Zo5m9Lnrh3FoOGD6h3NTq0rIta/+BVjcaoZYO4/paXGHNCzzel/3LCi4w+ogcAa19u5tIfPc83z+67tatn1m7a40yGraGmAU7S6HRhunmSKp1E2/Beey34w+S1fPzDbwxw37liFV06i1P+NUu/8NJVnHX6TvTs4f9brDG1HCaypZdL2hpq1kVNF6L7EXAU2UmyD0maFBEza7XPevrjXWs5cP9uDBqw8SMdP/FFbvvTWqZMHIKUfdnTpr/Kjbeu4dxvrWT1i8106gTbdxNnfHanelXdLKfG6aLWcgzuEGBeRDwNIGkC2QXrChngJvx+DWM+2uv19TvuWsulP3yeP9+0K927b/wx/OWWXV9/ftH3VtKzRycHN2s4vidDdiG6hSXrbV6crlG9/HIzf7rnZT52bI/X0/7jP1fw0trg6DHPctAHFvDFc5ZVKME6ivN/cxZX/m0cQ/fZhesW/ITRnz2i3lXqcLJZ1M5VLfVWyxZcVRenSxfAOx1g2JDGnNTt3r0Ty2fu8Ya0p+4f3uZ2F3ytX62qZJvpO6dcWe8qdHiNdKBvLVtwVV2cLiJ+GhEHR8TBA/rVP+KbWdua060D21rqrZZNpoeAEenCdM+S3Q3n5Bruz8y2Ap9sD0TEBklnApOBzsA1ETGjVvszs63Hs6hARNwO3F7LfZjZ1hUhNjjAmVlRbfNdVDMrJo/BmVmhOcCZWSH5ODgzK7T2Og5O0j8lPSHpMUkPp7S+kqZImpse+5TkPy9dvGOOpKPbKt8BzsxyiYANzZ2qWqr0/og4ICJabh94LjA1IkYAU9M6kkaSHU+7LzAauCpd1KMsBzgzy63Gl0s6Hhifno8HTihJnxAR6yJiPjCP7KIeZTnAmVkuLWNw7RTgArhT0iPpvHSAQRGxBCA9DkzpuS/g4UkGM8stqm+d9W8ZW0t+GhE/LVl/T0QsljQQmCJpdoWyqrqARykHODPLLceJ9CtKxtbeJCIWp8dlkm4m63IulTQ4IpZIGgy0XGusqgt4lHIX1cxyiWifMThJPST1ankOfBB4EpgEjE3ZxgK3pOeTgDGSuqWLeIwAplXah1twZpaTaGqfWwIOAm5Ol/PvAlwXEXdIegiYKOk0YAFwIkBEzJA0keyq4BuAMyKiqdIOHODMLLccY3AVyoingbe1kr4SOLLMNuOAqm9c6wBnZrn4XFQzK67IxuEagQOcmeXWES5HXg0HODPLJdpvkqHmHODMLDd3Uc2ssNpjFnVrcIAzs1wiHODMrMB8mIiZFZbH4MyskALR7FlUMyuqBmnAOcCZWU6eZDCzQmuQJpwDnJnl5hacmRVSAM3NDnBmVkQBuAVnZkXl4+DMrLgc4MysmORJBjMrMLfgzKyQAsKzqGZWXI0R4BrjjFkz61iiyqUKkjpLelTSrWm9r6Qpkuamxz4lec+TNE/SHElHt1W2A5yZ5deOAQ44C5hVsn4uMDUiRgBT0zqSRgJjgH2B0cBVkjpXKtgBzszyaTnQt5qlDZJ2BT4E/Lwk+XhgfHo+HjihJH1CRKyLiPnAPOCQSuU7wJlZbhHVLUB/SQ+XLKdvUtQVwDlAc0naoIhYku0nlgADU/oQYGFJvkUprSxPMphZftXPoq6IiINbe0HSccCyiHhE0qgqymptpxU7wm0GOEkCTgH2iIiLJQ0Ddo6IaVVUyMwKSO1zHNx7gI9IOhbYHugt6VpgqaTBEbFE0mBgWcq/CBhasv2uwOJKO6imi3oV8C7gpLT+EvCj6t+DmRVKtRMMbQTBiDgvInaNiN3IJg/uiohTgUnA2JRtLHBLej4JGCOpm6TdgRFAxYZWNV3Ud0bEQZIeTZV6XtJ2VWxnZoVU3QTCFrgEmCjpNGABcCJARMyQNBGYCWwAzoiIpkoFVRPg1qep2ACQNIA3Dgia2bamnU/Vioi7gbvT85XAkWXyjQPGVVtuNV3U/wFuBgZKGgfcB3yn2h2YWQE1V7nUWZstuIj4jaRHyCKqgBMiYlYbm5lZURXpgpdp1vRl4A+laRGxoJYVM7OOq51mUWuumjG428hitsimcncH5pCdLmFm26KiBLiI2L90XdJBwOdrViMzs3aS+0yGiJgu6R21qMxTj3fn6F0OqEXRViPLznx3vatgOWy4/oF2KacwXVRJXy1Z7QQcBCyvWY3MrGML8pyqVVfVtOB6lTzfQDYmd2NtqmNmDaEILbh0gG/PiPj6VqqPmTWAhu+iSuoSERvSpIKZ2UaNHuDITmI9CHhM0iTgd8Dalhcj4qYa183MOqoCBLgWfYGVwBFsPB4uAAc4s22QogBdVLJzT78KPMnGwNaiQd6emdVEAWZROwM92YyraJpZsRWhBbckIi7eajUxs8ZRgADXGG1QM9u6CjIG1+oF58zMGr4FFxGrtmZFzKxxqANczLIavi+qmRWW74tqZvk1ehfVzKxVDTTJ4C6qmeXXDvdFlbS9pGmS/i5phqSLUnpfSVMkzU2PfUq2OU/SPElzJB3dVjUd4Mwsv3YIcMA64IiIeBtwADBa0qHAucDUiBgBTE3rSBpJdoPofYHRwFXpikdlOcCZWS4im0WtZqkkMmvSate0BHA8MD6ljwdOSM+PByZExLqImA/MAw6ptA8HODPLJzaecN/WAvSX9HDJcnppUZI6S3oMWAZMiYgHgUERsQQgPQ5M2YcAC0s2X5TSyvIkg5nlV/0kw4qIOLhsMRFNwAGSdgJulrRfhbJynxfvFpyZ5dc+Y3Abi4tYDdxNNra2VNJggPS4LGVbBAwt2WxXYHGlch3gzCy3HF3U8mVIA1LLDUk7AB8AZgOTgLEp21jglvR8EjBGUjdJuwMjyC7MW5a7qGaWX/scBzcYGJ9mQjsBEyPiVkn3AxMlnQYsAE4EiIgZkiYCM8lugHVG6uKW5QBnZvlE+5yLGhGPAwe2kr6SMhf7iIhxwLhq9+EAZ2b5NciZDA5wZpZbo5yq5QBnZvk5wJlZIeU8BKSeHODMLBfhLqqZFZgDnJkVlwOcmRWWA5yZFVIDXdHXAc7M8nOAM7OiapTbBjrAmVlu7qKaWTH5QF8zKzQHODMrIp/JYGaFpubGiHAOcGaWj8fgzKzI3EU1s+JygDOzonILzsyKq0ECnO+Lamb5pLtqVbNUImmopD9LmiVphqSzUnpfSVMkzU2PfUq2OU/SPElzJB3dVlXdgquRAbv245zxZ9J3551obg5u/9mfuPl/bq93tQwYtFNPxn1yNP16dScCbvjbE1z3l0cBOOnwAxhz2Ntoag7umTGfKybdC8Bnj3oHHz10P5qbm/nujXfzt9nP1PMt1FU7Hge3ATg7IqZL6gU8ImkK8GlgakRcIulc4FzgG5JGAmOAfYFdgD9J2rvSvVEd4GqkaUMTV3/tf5n36Hx26Lk9Vz38XR6Z8jgLZi2qd9W2eU3NwfduvofZi5bRvVtXJnz9FB6Y8wz9enVn1P578vHvXsv6DU307bkDAHvs3JfRB+3Dx/7f/zKwdw+uPvNf+ci3fkVzNEg/rRba4b1HxBJgSXr+kqRZwBDgeGBUyjYeuBv4RkqfEBHrgPmS5gGHAPeX24e7qDWy6rnVzHt0PgCvrHmVBbOepf+QvnWulQGseHEtsxctA+Dldet5eukqBu7YkxPf+zaumfIQ6zdkDYJVa14BYNT+e3LH9Dms39DEs6teZOHy1ew3fOe61b8jUFS3AP0lPVyynN5qedJuZDeBfhAYlIJfSxAcmLINARaWbLYopZXlFtxWMGj4APY6cHdmPzi33lWxTezStzdvGTKAJ555jq8cfxgH7TmELx33btZtaOKy39/DjAVLGbRjTx7/55LXt1m6eg0Dd+pZx1rXWb4DfVdExMGVMkjqCdwIfDkiXpRUNmuZ2pRVsxacpGskLZP0ZK320Qi277E937zha/z4K7/k5ZdeqXd1rMQO23Xl+6cdx6U3/YW1r75Gl06d6N29G6deNoHLf38Pl37mQ1nGVv6sYlvuntI+kwwAkrqSBbffRMRNKXmppMHp9cHAspS+CBhasvmuwOJK5deyi/orYHQNy+/wOnfpzAU3nM1d193LfTdPq3d1rESXTp247LTjuP3h2Ux9fB4AS19Yw9S/Z8+fXLCU5gj69NyBpavXMKhPr9e3HbRTT5a/sLYu9e4o2mkWVcAvgFkRcVnJS5OAsen5WOCWkvQxkrpJ2h0YAVT8w6pZgIuIe4BVtSq/EZz98y+yYPaz3Hj5rfWuim3iwpOP4umlq/j1n6e/nvbnx//BIXtnDYThA3aia+fOPL/mFf7yxNOMPmgfunbpzJC+vRk2oA9PPvNcvapef0E2yVDNUtl7gE8CR0h6LC3HApcAR0maCxyV1omIGcBEYCZwB3BGpRlU6ABjcGnQ8XSA7ele59q0n33f8xaO+tT7ePrxZ/jJ9EsBuOY/r2PaHx+tc83swD124cOHjOSpZ5dz/TmnAPCDW//KzQ88ycUnf5Abz/0k65ua+L/XTgbgH8+t5M5Hn+Lm8z9FU1Mz3/ndXdv2DCrtc5hIRNxH6+NqAEeW2WYcMK7afaiWYwlpZuTWiNivmvy91TfeqVbfl3VQy858d72rYDnMvf4yXl66sOwofjV69hkaB7z/rKry/vXmrz/S1iRDLdW9BWdmjcUXvDSz4opomAte1vIwkd+SHWG8j6RFkk6r1b7MbCuLKpc6q1kLLiJOqlXZZlZf7qKaWTEF0CBdVAc4M8uvMeKbA5yZ5ecuqpkVVqPMojrAmVk+HWSGtBoOcGaWS3agb2NEOAc4M8uvikshdQQOcGaWm1twZlZMHoMzs+JqnHNRHeDMLD93Uc2skKK6+y10BA5wZpafW3BmVliNEd8c4MwsPzU3Rh/VAc7M8gka5kDfWt4X1cwKSASK6pY2y2rlBvGS+kqaImlueuxT8tp5kuZJmiPp6LbKd4Azs/za576o0PoN4s8FpkbECGBqWkfSSGAMsG/a5ipJnSsV7gBnZvm1U4Arc4P444Hx6fl44ISS9AkRsS4i5gPzgEMqle8AZ2b5tIzBVbNAf0kPlyynV7GHQRGxBCA9DkzpQ4CFJfkWpbSyPMlgZrnlmEVd0Y43fm7thtUVm4luwZlZTlV2Tzf/YOClkgYDpMdlKX0RMLQk367A4koFOcCZWT5BrQPcJGBsej4WuKUkfYykbpJ2B0YA0yoV5C6qmeXXTsfBpRvEjyIbq1sEXABcAkxMN4tfAJwIEBEzJE0EZgIbgDMioqlS+Q5wZpZbe13wssIN4o8sk38cMK7a8h3gzCw/n2xvZoUUAU2Nca6WA5yZ5ecWnJkVlgOcmRVSAL4ng5kVU0B4DM7MiijwJIOZFZjH4MyssBzgzKyYtug8063KAc7M8gnAN50xs8JyC87MismnaplZUQWEj4Mzs8LymQxmVlgegzOzQorwLKqZFZhbcGZWTEE0VbwVQofhAGdm+fhySWZWaD5MxMyKKIBwC87MCil8wUszK7BGmWRQdKDpXknLgWfqXY8a6A+sqHclLJeifmfDI2LAlhQg6Q6yz6caKyJi9Jbsb0t0qABXVJIejoiD610Pq56/s2LoVO8KmJnVigOcmRWWA9zW8dN6V8By83dWAA5wW0FE1PWPRVKTpMckPSnpd5K6b0FZv5L08fT855JGVsg7StK7N2Mf/5RU7SB2TdT7O7P24QC3bXglIg6IiP2A14AvlL4oqfPmFBoRn4uImRWyjAJyBziz9uIAt+25F9grta7+LOk64AlJnSVdKukhSY9L+jyAMj+UNFPSbcDAloIk3S3p4PR8tKTpkv4uaaqk3cgC6VdS6/EwSQMk3Zj28ZCk96Rt+0m6U9Kjkq4GtHU/EisqH+i7DZHUBTgGuCMlHQLsFxHzJZ0OvBAR75DUDfirpDuBA4F9gP2BQcBM4JpNyh0A/Aw4PJXVNyJWSfoJsCYivpfyXQdcHhH3SRoGTAb+BbgAuC8iLpb0IeD0mn4Qts1wgNs27CDpsfT8XuAXZF3HaRExP6V/EHhry/gasCMwAjgc+G1ENAGLJd3VSvmHAve0lBURq8rU4wPASOn1BlpvSb3SPj6Wtr1N0vOb+T7N3sABbtvwSkQcUJqQgsza0iTgSxExeZN8x5KdX12JqsgD2ZDIuyLilVbq4iPOrd15DM5aTAa+KKkrgKS9JfUA7gHGpDG6wcD7W9n2fuB9knZP2/ZN6S8BvUry3Qmc2bIiqSXo3txH+0sAAACGSURBVAOcktKOAfq027uybZoDnLX4Odn42nRJTwJXk7XwbwbmAk8APwb+sumGEbGcbNzsJkl/B65PL/0B+GjLJAPwH8DBaRJjJhtncy8CDpc0nayrvKBG79G2MT4X1cwKyy04MyssBzgzKywHODMrLAc4MyssBzgzKywHODMrLAc4Myus/w/PJxxPkUqY1wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_confusion_matrix(test_Y[1], np.round(type_pred), title='Wine Type', labels = [0, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GW91ym8P2I5y"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQwAAAEWCAYAAACTwaluAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAcVklEQVR4nO3df5wcdZ3n8dc7vySJQMCEBQJJQDF3Yg6CIwS5dXFZ5QHLYmR5aHA5F26XGI51T9fNrawuh3vc6T7iuqLsESKoIBoVCDnOBaKsitytoJNfBBBukR8mE34MP0KEDBCSz/1RNaGn0z1TPdNV3V39fj4e85jub9V0faam+j1V1d/6liICM7MsxrW6ADPrHA4MM8vMgWFmmTkwzCwzB4aZZebAMLPMHBhmlpkDwzKT9GLF125JAxXP/6jV9Vn+5I5bNhqSHgP+NCLuaHUtVhzvYdiYSHqDpOckzatoOyjd+5gh6WRJWyT9taRnJD1WuTeS/vwXJP1a0lOSlkua3JrfxkbiwLAxiYhXgO8A51Y0nwPcERH96fODgenATOCPgRWS5qbT/g54K3As8JZ0nksKKN1GwYFhzXAt8GFJg9vTfwC+WTXP30TEKxFxJ/BPwAclCbgA+EREPBcRvwH+B7CoqMKtMRNaXYB1voi4R9JLwO9IeoJkT+GWilmej4iXKp4/DhwKzACmAGuT7ABAwPj8q7bRcGBYs1xLcljyJHBjRLxcMe0ASVMrQmMWcB/wDDAAHB0RfYVWa6PiQxJrlm8CHyAJjetqTP+spEmSfhs4A7ghInYDXwX+QdJBAJJmSjq1qKKtMQ4Ma4qI2AKsAwK4q2ryk8DzwFbgW8CSiHgwnfZXwMPA3ZK2A3cAc7G25H4Y1jSSvgZsjYjPVLSdDFwfEYe1rDBrGp/DsKaQNAc4C5jf2kosTz4ksTGT9N9ITmIui4hHW12P5ceHJGaWmfcwzCyzjjuHMX369JgzZ06ryzBrim07drL5+R1MnTSBOdOnMO71DmwttXbt2mciYkZ1e8cFxpw5c+jt7W11GWZjtnp9H3/xvQ2cdcSbuOa8HqZMap+3o6THa7X7kMSsBQbD4oQ2DIvhODDMCtapYQEODLNCdXJYgAPDrDCdHhbgwDArRBnCAjrwU5JaVq/vY9mah9i6bYBDp01m6alzWTh/ZqvLMgPKExZQgsBYvb6Pi1dtYmDnLgD6tg1w8apNAA4Na7kyhQXkeEgiaa6kDRVf2yV9vGqekyW9UDFPw2M5Llvz0J6wGDSwcxfL1jw0xt/AbGzKFhaQ4x5GRDxEMrArksYDfcDNNWa9KyLOGO1ytm4baKjdrAhlDAso7qTnKcCvIqJm77GxOHRa7RHp67Wb5a2sYQHFBcYiYGWdaSdK2ijpNklH15pB0mJJvZJ6+/v7h0xbeupcJk8cOmbs5InjWXqqB22y4pU5LKCAy9slTSIZmu3oiHiqatp+wO6IeFHS6cDlEXHUcK/X09MT1deS+FMSawdlCgtJayOip7q9iN/oNGBddVgARMT2ise3SvqfkqZHxDONLGDh/JkOCGupMoXFcIo4JDmHOocjkg5Ob2aDpOPTep4toCazpumWsICc9zAkTQHeC3y0om0JQEQsB84GLpT0Gsn9KRaFhwCzDtJNYQE5B0ZE7ADeVNW2vOLxFcAVedZglpduCwvwtSRmo9KNYQEODLOGdWtYgAPDrCHdHBbgwDDLrNvDAhwYZpk4LBIODLMROCxe58AwG4bDYqju/u1LxNfTNJ/DYm9eAyXgUceaz2FRmw9JSsCjjjWXw6I+r4kS8KhjrxvroZnDYnjewygBjzqWGDw069s2QPD6odnq9X2Zf95hMTwHRgl41LHEWA7NHBbZeK2UwOAud7d/SjLaQzOHRXbew7DSGM2hmcOiMQ6MEli9vo+lN2wccuy+9IaNmY/dy6LRQzOHReMcGCVw6S33s3P30IHKdu4OLr3l/hZV1BoL58/kc2fNY+a0yQiYOW0ynztrXs1DM4fF6HgtlcC2gZ0NtZdZlgGhHRaj5z0M6yoOi7FxYFjXcFiMnQOjBKZNnthQezdyWDSHA6MELj3zaCaO05C2iePEpWfWvPNk13FYNI8DowQWzp/Jh44/nPHJPaEYL/Gh4w/vuo5btTgsmiu3tSdpLvDdiqYjgUsi4ksV8wi4HDgd2AGcFxHr8qqprFav7+OmtX3sSu8BtSuCm9b20TP7wK4LjcqLz6ZNmci2gZ0scFg0TW57GBHxUEQcGxHHAu8gCYSbq2Y7DTgq/VoMXJlXPWXmy9sT1RefPb9jJwIWzj/UYdEkRR2SnAL8KiIer2p/P3BdJO4Gpkk6pKCaSsOXtydqBefugC//88Mtqqh8igqMRdS+IfNMYHPF8y1p2xCSFkvqldTb39+fU4mdy5e3Jxyc+cs9MCRNAs4Ebqg1uUbbXjdjjogVEdETET0zZsxodokdz5e3J6ZNqf0xcrcFZ56KOLA7DVgXEU/VmLYFOLzi+WHA1gJqKhVf3p6cv9g2kJyzqPyPM3Gcui4481TEIck51D4cAbgF+IgSC4AXIuKJAmqyEhn86PQtM96417TBT46sOXINDElTgPcCqyralkhakj69FXgEeBj4KvCf8qynrLr58vbKfhZ9z+/Y63h2d8Cnb97UktrKKNfAiIgdEfGmiHihom15RCxPH0dEXBQRb46IeRHRm2c9ZdWtl7dXd8rasXN3zfleenVXzXZrnHt6lkA3Xt7uHpytUYq17Lt+dReHRet0/Jr2Xb+6i8OitTr+kMTdoruHw6L1Oj4w3LuvOzgs2kPHB4a7RZefw6J9dHxguFt0uTks2kvHr313i4apk8bX7GswddL4GnN3DodF+ynFXyDL0PJltqNOx6R67Z3AYdGeOv6QxOpfpVmvvd05LNqXA6MEXny5do/Oeu3tzGHR3hwYJVDnEoq67e3KYdH+HBjWFhwWncGBUQLjao1bNkx7u3FYdA4HRgl8+IRZDbW3E4dFZ/FfpwQuWzgPgJX3bGZXBOMlzjnh8D3t7cph0Xm8h1ESPbMP5OD990HAwfvvQ8/sA1td0rAcFp3Jf6USWL2+j49/d8Oe533bBvY8b8cObQ6LzuU9jBJYesOGhtpbyWHR2RwYJdAp/TAcFp3PgWGFcFiUgwPDcuewKA8HhuXKYVEuDgzLjcOifHL9C0qaBlwNvJ3klpf/MSJ+VjH9ZOB/AY+mTasi4m8bXY5vM9B+HBbllPdf8XLg9og4O72L+5Qa89wVEWeMdgG+zUD7cViUV26HJJL2A94NXAMQEa9GxLZmL8e3GWgvDotyy/McxpFAP/B1SeslXS1pao35TpS0UdJtko6u9UKSFkvqldTb398/ZJpvM9A+HBbll2dgTACOA66MiPnAS8CnquZZB8yOiGOArwCra71QRKyIiJ6I6JkxY8aQaftMrP0r1Gu3fDgsukOe76otwJaIuCd9fiNJgOwREdsj4sX08a3AREnTG1nIK6/V7s5Yr92az2HRPXILjIh4EtgsafAGIacAD1TOI+lgSUofH5/W82wjy9kdjbVbczksukvef92PAd9KPyF5BDhf0hKAiFgOnA1cKOk1YABYFBF+q3cIh0X3yfUvHBEbgJ6q5uUV068ArsizBsuHw6I7+cygNcxh0b0yBYakN0t6Q/r4ZEl/nvbitC7jsOhuWfcwbgJ2SXoLSUesI4Bv51aVtSWHhWUNjN0R8RrwAeBLEfEJ4JD8ysqu3kj6HTLCfsdwWBhkD4ydks4B/hj4ftrWFjfurPeRij9qaR6HhQ3KGhjnAycC/z0iHpV0BHB9fmVZu3BYWKVMf/2IeAD484rnjwKfz6soaw8OC6uWaQuQdBJwKTA7/RkBERFH5leatZLDwmrJuhVcA3wCWAvsGmFe63AOC6sn65bwQkTclmsl1hYcFjacrFvDjyUtA1YBrww2RsS6XKqylnBY2EiybhEnpN8rrwsJ4HebW461isPCssj6Kcl78i5ktMZL7Kpxget4uetWVg4LyyrrtST7S/ri4DB5kv5e0v55F5fFOScc3lC7DeWwsEZk7bj1NeA3wAfTr+3A1/MqyorhsLBGZQ2MN0fEf42IR9Kvz5IM8ttyK+/Z3FC7JRwWNhpZA2NA0r8ffJJ25GqLYblrnb8Yrt0cFjZ6WbeUC4Fr0/MWAp4DzsurKMuPw8LGIuunJBuAY9KbExER23OtynLhsLCxGnaLkXRuRFwv6S+q2gGIiC/mWJs1kcPCmmGkrWbwTmX71pjmkwQdwmFhzTLslhMRV6UP74iI/1s5LT3xaW3OYWHNlPVTkq9kbLM247CwZhrpHMaJwLuAGVXnMfYDxudZmDWHw8KaaaQ9jEnAG0mCZd+Kr+0kdy0blqRpkm6U9KCkX6YBVDldkr4s6WFJ90o6rt5r2eg4LKyZRjqHcSdwp6RvRMTjo3j9y4HbI+Ls9HaJU6qmnwYclX6dAFzJ61fGWhM4LKyZsp7DuLryxkWSDpC0ZrgfSPtsvJtktC4i4tWI2FY12/uB6yJxNzBNUlvcvsDM9pY1MKZXvtkj4nngoBF+5kigH/i6pPWSrpY0tWqemUDlRR9b0rYhJC0evFK2v78/Y8nWbY46qHrzGr7dGpf5RkaSZg0+kTSbkfthTACOA66MiPnAS8CnquapNWjFXq8bESsioiciembMmJGxZOs2O17d3VC7NS7rAe6ngf8j6c70+buBxSP8zBZgS0Tckz6/kb0DYwtQOXDFYcDWjDUZST8LS/Rtq309ZL12a1ymPYyIuJ1kb+G7wPeAd0TEsOcwIuJJYLOkuWnTKcADVbPdAnwk/bRkAclgw0808gt0s8FOWZYYV2eQtXrt1riR+mH8m4h4sOLjzsH//rMkzcowCPDHgG+ln5A8ApwvaQlARCwHbgVOBx4GdpDcYc0yqOzB+bNHnm11OW1hd52D5Hrt1riRDkk+CVwA/H2NaSMOApxe5dpT1by8YnoAF41cplWq7u79tkuG3dkza5qR+mFckH5v20GAu42vDbFWGumQ5KzhpkfEquaWY8NxWFirjbTF/UH6/SCSa0p+lD5/D/ATkhsbWQEcFtYORjokOR9A0veBtw1+gpH2xvzH/MszcFhY+8jacWtO1cedTwFvzaEeq+KwsHaSdev7SXrtyEqST0cWAT/OrSoDHBbWfrIOAvxnkj5A0sMTYEVE3JxfWeawsHbUyFa4DvhNRNwhaYqkfSPiN3kV1s0cFtaust5b9QKSa0EGx/icCazOq6hu5rCwdpb1pOdFwEkkI20REf/KyJe3W4McFmNT75IRX0rSPFkD45WIeHXwiaQJ+DYDTeWwGLt6G6Q31ObJGhh3SvprYLKk9wI3AP87v7K6i8PCOkXWwPgrktGzNgEfJbnK9DN5FdVNHBbWSUbcOiWNA+6NiLcDX82/pO7hsLBOM+IeRkTsBjZWDtFnY+ewsE6UdSs9BLhf0s9JxuYEICLOzKWqknNY5OOAKRN5fsfOmu3WHFm31M/mWkUXcVjk58WX9w6L4dqtcSONh7EPsAR4C8kJz2si4rUiCisjh0W+dtYZHLxeuzVupHMY15IMsbeJ5C5ltYbqswwcFlYGI221b4uIeQCSrgF+nn9J5eOwKMY41R7w16OGN89Iexh7Dv58KDI6DovinHjkgQ21W+NG2nqPkbQ9fSySnp7b08cREfvlWl2Hc1gU64Enal88Xa/dGjfSEH3jiyqkbBwWxav1kepw7da4rF3DrQEOCyurXAND0mOSNknaIKm3xvSTJb2QTt8g6ZI86ymCw6J1pk2u3UGrXrs1rog9jPdExLERUX0HtEF3pdOPjYi/LaCe3DgsWuvSM49mYtVHIhPHiUvPPLpFFZWPt+gmcVi03sL5MwFYtuYhtm4b4NBpk1l66tw97TZ2eW/VAfxAUgBXRcSKGvOcKGkjyY2e/zIi7q+eQdJiYDHArFntdw2cw6J9LJw/0wGRo7y37JMiYqukg4AfSnowIn5aMX0dMDsiXpR0Osk4oUdVv0gaNCsAenp62moAJYeFdZNcz2FExNb0+9PAzcDxVdO3R8SL6eNbgYmSpudZUzM5LKzb5BYYkqZK2nfwMfA+4L6qeQ6WpPTx8Wk9z+ZVUzM5LKwb5bmV/xZwc5oHE4BvR8TtkpYARMRy4GzgQkmvAQPAoohoq0OOWhwW1q1y29Ij4hHgmBrtyyseXwFckVcNeXBYWDdzT88GOCys2zkwMnJYmLnjViYOi86xen2fO27lyFv+CBwWnWP1+j4uXrWJgZ27AOjbNsDFqzYBODSaxIckw3BYdJZlax7aExaDBnbuYtmah1pUUfk4MOpwWHSerdsGGmq3xjkwanBYdKZDp01uqN0a58Co4rDoXEtPncvkiUMHiZs8cTxLT53boorKx++GCg6LzubL2/Pnd0TKYVEOvrw9Xz4kwWFhllXXB4bDwiy7rg4Mh4VZY7o2MBwWZo3rysBwWJiNTtcFhsPCbPS6KjAcFmZj0zWB4bAwG7uuCAyHhVlzlD4wHBZmzVPqwHBYmDVXqd9BDovu4yH68lXqd5HDort4iL785XpIIukxSZskbZDUW2O6JH1Z0sOS7pV0XDOX3y1hMT65WVTm9rLyEH35K+Icxnsi4tiI6Kkx7TSSmy8fRXJ39iubueBuCAuAc044vKH2svIQfflr9UnP9wPXReJuYJqkQ1pcU8e5bOE8zl0wa88exXiJcxfM4rKF81pcWbE8RF/+8v4XHMAPJAVwVUSsqJo+E9hc8XxL2vZE5UySFpPsgTBr1qz8qu1gly2c13UBUW3pqXOHnMMAD9HXbHkHxkkRsVXSQcAPJT0YET+tmF7rIHuvmzGnQbMCoKenp+1v1myt4SH68pdrYETE1vT705JuBo4HKgNjC1B5oH0YsDXr669e39eMMq1EPERfvnI7hyFpqqR9Bx8D7wPuq5rtFuAj6aclC4AXIuIJMhjslGWJ1ev7OOnzP+KIT/0TJ33+Rw5Ty0Weexi/Bdys5ETcBODbEXG7pCUAEbEcuBU4HXgY2AGcn+WFK3tw/uyRZ3MpvpO4/4EVJbfAiIhHgGNqtC+veBzARY287rYdO4f04HzbJWvGXmyHG67/gQPDmqnVH6s2bPPzO9yDs4r7H1hROi4wpk6aMCQs3MsR9p88saF2s9HquMCYM33KkD2LXVH7U9Z67WVULxu7KDOtIB0XGOOq3gXT6vwXrddeRtt27Gyo3Wy0Oi4wqvm/q7tEW3E6PjD839V3LbfidPzHDPtPnsi2gb3DoZtO+LlLtBWl4wPDhyQJd4m2IviQxMwy6/jA8Ak/s+J0fGD4hJ9ZcTr+HIZP+JkVp+MDA3zCz6woHX9IYmbFcWCYWWYODDPLzIFhZpmV4qSn76dpVoyODwyPZ2lWnI4/JPH9NM2K0/GB4fEszYrT8YHha0nMitPxgeFrScyK0/EnPX0tiVlxct/DkDRe0npJ368x7WRJL0jakH5dMppl9D7+HE++8DIBPPnCy/Q+/tyY6zazvRWxh/GfgV8C+9WZfldEnDHaF//M6k1cf/ev9zzfFbHn+WUL5432Zc2shlz3MCQdBvw+cHVey1h5z+aG2s1s9PI+JPkS8F+A3cPMc6KkjZJuk3R0rRkkLZbUK6m3v79/yDTfyMisOLkFhqQzgKcjYu0ws60DZkfEMcBXgNW1ZoqIFRHRExE9M2bMGDLNt0o0K06eexgnAWdKegz4DvC7kq6vnCEitkfEi+njW4GJkqY3spBzTji8oXYzG73cAiMiLo6IwyJiDrAI+FFEnFs5j6SDpWRXQNLxaT3PNrKcyxbO49wFs/bsUYyXOHfBLJ/wNMtB4f0wJC0BiIjlwNnAhZJeAwaARRGNn3y4bOE8B4RZATSK92dL9fT0RG9vb6vLMCs1SWsjoqe6veO7hptZcRwYZpaZA8PMMnNgmFlmHXfSU1I/8HidydOBZwosp92W7xrap4ZWL3+sNcyOiBnVjR0XGMOR1FvrzG63LN81tE8NrV5+XjX4kMTMMnNgmFlmZQuMFV2+fHANg1pdQ6uXDznUUKpzGGaWr7LtYZhZjhwYZpZZRwaGpMckbUoHDt7rSjQlvizpYUn3Sjqu4OU3ZXDjEWqYJulGSQ9K+qWkE6um57oOMtaQ63qQNLfitTdI2i7p41Xz5LYeMi6/iG3hE5Lul3SfpJWS9qma3rx1EBEd9wU8BkwfZvrpwG2AgAXAPQUv/2Tg+zmvg2uBP00fTwKmFbkOMtaQ+3qoWNZ44EmSDkeFrocRlp/rOgBmAo8Ck9Pn3wPOy2sddOQeRgbvB66LxN3ANEmHtLqoZpG0H/Bu4BqAiHg1IrZVzZbrOshYQ5FOAX4VEdW9gIvaFuotvwgTgMmSJgBTgK1V05u2Djo1MAL4gaS1khbXmD4TqBw2fEvaVtTyIcPgxmNwJNAPfF3JPV+uljS1ap6810GWGiDf9VBpEbCyRnve62Gk5UOO6yAi+oAvAL8GngBeiIgfVM3WtHXQqYFxUkQcB5wGXCTp3VXTa40A3MzPj0dafqbBjcdgAnAccGVEzAdeAj5VNU/e6yBLDXmvBwAkTQLOBG6oNblGW1P7Eoyw/FzXgaQDSPYgjgAOBaZKOrd6tho/Oqp10JGBERFb0+9PAzcDx1fNsgWoHAX4MPbeTctt+dGEwY1HsAXYEhH3pM9vJHnzVs+T2zrIUkMB62HQacC6iHiqTp15rodhl1/AOvg94NGI6I+IncAq4F1V8zRtHXRcYEiaKmnfwcfA+4D7qma7BfhIenZ4Aclu2hNFLV9NGNx4OBHxJLBZ0uAdp08BHqiaLbd1kLWGvNdDhXOofziQ63oYafkFrINfAwskTUmXcwrJnQYrNW8d5HX2NsezwkcCG9Ov+4FPp+1LgCXpYwH/CPwK2AT0FLz8P0unbQTuBt6Vw3o4FugF7iXZzT2gqHXQQA1FrIcpJG/A/SvaClsPGZZfxDr4LPAgyT+ubwJvyGsduGu4mWXWcYckZtY6Dgwzy8yBYWaZOTDMLDMHhpll5sDoMpLeVHHl5JOS+iqeT2rC618q6XNVbcdKqu4bUP0zfznWZVv+Cr8Zs7VWRDxL0n8CSZcCL0bEFwanS5oQEa+NYRErSa6MvLiibRHw7TG8prUJ72EYkr4h6YuSfgz8XfV//HSchTnp43Ml/TzdI7lK0vjK14qIh4Btkk6oaP4g8B1JF0j6RXoh1k2SptSo5SeSetLH0yU9lj4eL2lZ+vP3Svpo2n6IpJ+m9dwn6bebunJsCAeGDXor8HsR8cl6M0j6t8CHSC6+OxbYBfxRjVlXkuxVkHZFfjYi/hVYFRHvjORCrF8Cf9JAfX9C0qX5ncA7gQskHQF8GFiT1nMMsKGB17QG+ZDEBt0QEbtGmOcU4B3AL9LLIyYDT9eY7zvAv0j6JEMv+367pMuAacAbgTUN1Pc+4N9JOjt9vj9wFPAL4GuSJgKrI8KBkSMHhg16qeLxawzd+xwc8k3AtRFReX5iLxGxOT2U+B3gD4HBofu+ASyMiI2SziMZjapa5bIrh5oT8LGI2Ctk0uEFfh/4pqRlEXHdcPXZ6PmQxGp5jPRSdSXjPx6Rtv8zcLakg9JpB0qaXec1VgL/QDIK1Za0bV/giXRvoNahzOCy35E+PruifQ1wYfqzSHpreuXwbODpiPgqyehfTR+71F7nwLBabgIOlLQBuBD4fwAR8QDwGZLRxu4FfgjUG+rtBuBoksOTQX8D3JP+3IN1fu4LJMHwLyQ3Ex50Ncnl8+sk3QdcRbKHfDKwQdJ6kr2Zyxv6Ta0hvlrVzDLzHoaZZebAMLPMHBhmlpkDw8wyc2CYWWYODDPLzIFhZpn9f9lfkucfPadmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter_plot = plot_diff(test_Y[0], quality_pred, title='Type')"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "name": "exercise-answer.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}