OVER01. Naive Oversampling, SMOTE and ADASYN.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imbalanced classes in a dataset can skew model boundaries (e.g. SVC, SVR) to favour the majority class\n",
    "# There is a need to oversample the minority class (e.g. if only 0.5% of response is 1 and 99.5% is 0)\n",
    "# Can do naive sampling of 1s by repeating them in dataset until they end up 50-50% for 0s and 1s\n",
    "# By giving the 1s more \"say\" through more points, their error cannot be neglected and model will prioritize them more\n",
    "# This is similar to using a unique RMSE or accuracy metric that weights the 1 more heavily by a ratio of beta = no of 0/no of 1s\n",
    "# However some models might work differently for error weighing metric vs naive sampling\n",
    "\n",
    "# Besides random naive sampling, there's always cool methods of making synthetic data\n",
    "\n",
    "# SMOTE - Synthetic Minority Over-Sampling Technique\n",
    "# For each rare datapoint of the minority, SMOTE finds the nearest neighbors\n",
    "# Drawing a line (or line traversing multi dimensions) between the current point and nearest neighbor, a new point is created on that imaginary point\n",
    "# It is a kind of interpolation that gets the middle ground between the points\n",
    "# e.g. a=0, b=1, c=2, y=1 (+ve rare response) is found to have nearest neighbor of a=0.2, b=-3, c=2, y=1 (+ve response)\n",
    "# Nearest neighbor is found using RMS distance of features a,b,c and y is both of 1\n",
    "# Then a possible new SMOTE point is a=0.1, b=0, c=2 - notice how the points lie on the 3D line between the two points \n",
    "# Advantage of SMOTE: Synthetic observations are less likely to overfit than repeating data through naive samples\n",
    "# While no data loss is present if you do oversampling of 1s (and not undersample the 0s to a small number), there is no new info\n",
    "# SMOTE creates new info with the assumption that future 1s would have similar features like the 0s\n",
    "# However if 1 data points are too little, data quality bad or interpolation not realistic due to circumstances, then model might not work on future data\n",
    "\n",
    "# Adaptive Synthetic (ADASYN) - alternative to SMOTE\n",
    "# While SMOTE is good in creating synthetic points, it tends to create points in between current points\n",
    "# Hence the density of points will end up concentrated inside the widest permimeter of the area (or hyperdimensional volume)\n",
    "# e.g. if you have a sphere of points of coordinates (a,b,c), all lines will be in the sphere or near its surface\n",
    "# Hence the points spawned on the line will not be concentrated on border regions with the 0s where a clear distinction is important for models\n",
    "# A better info metric might be to create points with priority at CHALLENGING region of 1s and 0s \n",
    "# Some regions might only have a few 1s and so it needs help to seperate the points more clearly by putting more 1s in neighborhood\n",
    "# This may work very well for boundary based models like SVC, or random forest\n",
    "\n",
    "# First, ADASYN finds the ratio b of 0s vs 1s. If 0 is 10 times of 1, then b=10\n",
    "# G = (ml-ms)*b, where G is total number of synthetic points to create and ms and ml are weights to be set?\n",
    "# Next, ADASYN finds k nearest neighbors, e.g. if there are 5 neighbors near the current 1 point, k=5\n",
    "# Out of the k neighbors, the number of points that are 0s (the majority class) will be determined\n",
    "# A ri value is calculated based on (no. of majority points in k neighbors)/no. of k neighbors\n",
    "# e.g. if there are 5 neighbors and 3 are 0s (majority class), ri = 3/5\n",
    "# Each point of response 1 will be looped through to find k neighbors and ri value\n",
    "# The higher rᵢ, the more dominant the majority class and the more difficult the neighborhood for that point is to learn for your classifier.\n",
    "# ADASYN normalizes the r values for all minority observations so that their sum is one.\n",
    "# ADASYN computes the number of synthetic observations to generate in each neighborhood, i.e. Gi = G*ri\n",
    "# where G is total number to be made, ri is ratio for the neighborhood of point i, Gi is num of points for the neighborhood\n",
    "# For each point that still has a rare (response 1) neighbor nearby, the usual linear interpolation is done\n",
    "# si = xi + (xzi - xi) * lambda, where lambda is random factor from 0 to 1, xi is current point, xzi is neighbor and si is new point\n",
    "# Alternatively, we can mark out a plane from 3 points (of response 1) and create a point any point on that plane\n",
    "\n",
    "# As with most algorithms, there are several extensions of SMOTE. \n",
    "# These aim to improve SMOTE by adding to its functionality or lessening its weaknesses.\n",
    "# Extension of SMOTE: (examples)\n",
    "# BorderlineSMOTE: Instead of oversampling between all minority observations, BorderlineSMOTE aims to increase the number of minority observations that border majority observations. \n",
    "# The goal here is to allow the classifier to be able to distinguish between these borderline observations more clearly.\n",
    "# ADASYN focuses more on challenging areas with little 1s, which need not be the boundaries. But BorderlineSMOTE is more border based\n",
    "# SVMSMOTE: SVMSMOTE, as its name implies, utilizes the Support Vector Machine algorithm to generate new minority observations close to the border between the majority and minority classes.\n",
    "\n",
    "# Different algos have different ways of generating the points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>102087</td>\n",
       "      <td>68047.0</td>\n",
       "      <td>-1.919985</td>\n",
       "      <td>2.161635</td>\n",
       "      <td>-0.007638</td>\n",
       "      <td>-0.090821</td>\n",
       "      <td>-0.684380</td>\n",
       "      <td>-0.362199</td>\n",
       "      <td>-0.495096</td>\n",
       "      <td>1.460793</td>\n",
       "      <td>-0.917117</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.109496</td>\n",
       "      <td>-0.470316</td>\n",
       "      <td>0.156857</td>\n",
       "      <td>-0.016690</td>\n",
       "      <td>-0.062102</td>\n",
       "      <td>0.097150</td>\n",
       "      <td>0.108513</td>\n",
       "      <td>0.007307</td>\n",
       "      <td>9.42</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>69593</td>\n",
       "      <td>53495.0</td>\n",
       "      <td>0.937565</td>\n",
       "      <td>-0.251527</td>\n",
       "      <td>-0.019672</td>\n",
       "      <td>0.992319</td>\n",
       "      <td>-0.123135</td>\n",
       "      <td>0.024872</td>\n",
       "      <td>0.096008</td>\n",
       "      <td>0.125683</td>\n",
       "      <td>-0.046874</td>\n",
       "      <td>...</td>\n",
       "      <td>0.144192</td>\n",
       "      <td>0.104474</td>\n",
       "      <td>-0.215257</td>\n",
       "      <td>-0.330250</td>\n",
       "      <td>0.531800</td>\n",
       "      <td>-0.304207</td>\n",
       "      <td>-0.007529</td>\n",
       "      <td>0.022114</td>\n",
       "      <td>130.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>199573</td>\n",
       "      <td>133028.0</td>\n",
       "      <td>-1.166955</td>\n",
       "      <td>0.197206</td>\n",
       "      <td>-0.166187</td>\n",
       "      <td>-3.259610</td>\n",
       "      <td>1.146332</td>\n",
       "      <td>4.990452</td>\n",
       "      <td>-1.239286</td>\n",
       "      <td>-2.106628</td>\n",
       "      <td>1.466259</td>\n",
       "      <td>...</td>\n",
       "      <td>2.613893</td>\n",
       "      <td>-2.159352</td>\n",
       "      <td>-0.567234</td>\n",
       "      <td>0.734404</td>\n",
       "      <td>1.422567</td>\n",
       "      <td>0.498377</td>\n",
       "      <td>0.303421</td>\n",
       "      <td>0.077623</td>\n",
       "      <td>200.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6394</td>\n",
       "      <td>7594.0</td>\n",
       "      <td>1.154254</td>\n",
       "      <td>0.283372</td>\n",
       "      <td>0.835510</td>\n",
       "      <td>1.468512</td>\n",
       "      <td>-0.510746</td>\n",
       "      <td>-0.676945</td>\n",
       "      <td>-0.144957</td>\n",
       "      <td>-0.182303</td>\n",
       "      <td>1.460631</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.213593</td>\n",
       "      <td>-0.438333</td>\n",
       "      <td>0.020777</td>\n",
       "      <td>0.295094</td>\n",
       "      <td>0.368346</td>\n",
       "      <td>-0.519646</td>\n",
       "      <td>-0.000689</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>36.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>130799</td>\n",
       "      <td>79435.0</td>\n",
       "      <td>-0.061008</td>\n",
       "      <td>-0.744399</td>\n",
       "      <td>1.800945</td>\n",
       "      <td>-2.150292</td>\n",
       "      <td>-1.040646</td>\n",
       "      <td>0.448124</td>\n",
       "      <td>-0.344005</td>\n",
       "      <td>-0.007536</td>\n",
       "      <td>-2.311161</td>\n",
       "      <td>...</td>\n",
       "      <td>0.059198</td>\n",
       "      <td>0.617976</td>\n",
       "      <td>-0.030908</td>\n",
       "      <td>-0.313231</td>\n",
       "      <td>-0.141417</td>\n",
       "      <td>-0.142070</td>\n",
       "      <td>-0.096524</td>\n",
       "      <td>-0.202511</td>\n",
       "      <td>79.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Time        V1        V2        V3        V4        V5        V6  \\\n",
       "102087   68047.0 -1.919985  2.161635 -0.007638 -0.090821 -0.684380 -0.362199   \n",
       "69593    53495.0  0.937565 -0.251527 -0.019672  0.992319 -0.123135  0.024872   \n",
       "199573  133028.0 -1.166955  0.197206 -0.166187 -3.259610  1.146332  4.990452   \n",
       "6394      7594.0  1.154254  0.283372  0.835510  1.468512 -0.510746 -0.676945   \n",
       "130799   79435.0 -0.061008 -0.744399  1.800945 -2.150292 -1.040646  0.448124   \n",
       "\n",
       "              V7        V8        V9  ...       V21       V22       V23  \\\n",
       "102087 -0.495096  1.460793 -0.917117  ... -0.109496 -0.470316  0.156857   \n",
       "69593   0.096008  0.125683 -0.046874  ...  0.144192  0.104474 -0.215257   \n",
       "199573 -1.239286 -2.106628  1.466259  ...  2.613893 -2.159352 -0.567234   \n",
       "6394   -0.144957 -0.182303  1.460631  ... -0.213593 -0.438333  0.020777   \n",
       "130799 -0.344005 -0.007536 -2.311161  ...  0.059198  0.617976 -0.030908   \n",
       "\n",
       "             V24       V25       V26       V27       V28  Amount  Class  \n",
       "102087 -0.016690 -0.062102  0.097150  0.108513  0.007307    9.42      0  \n",
       "69593  -0.330250  0.531800 -0.304207 -0.007529  0.022114  130.00      0  \n",
       "199573  0.734404  1.422567  0.498377  0.303421  0.077623  200.00      0  \n",
       "6394    0.295094  0.368346 -0.519646 -0.000689  0.030078   36.00      0  \n",
       "130799 -0.313231 -0.141417 -0.142070 -0.096524 -0.202511   79.00      0  \n",
       "\n",
       "[5 rows x 31 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# importing packages\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import scipy.stats as stats\n",
    "import sklearn\n",
    "import imblearn\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "plt.style.use('ggplot')\n",
    "\n",
    "# reading in CSV\n",
    "df = pd.read_csv('creditcard.csv')\n",
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 284807 entries, 0 to 284806\n",
      "Data columns (total 31 columns):\n",
      "Time      284807 non-null float64\n",
      "V1        284807 non-null float64\n",
      "V2        284807 non-null float64\n",
      "V3        284807 non-null float64\n",
      "V4        284807 non-null float64\n",
      "V5        284807 non-null float64\n",
      "V6        284807 non-null float64\n",
      "V7        284807 non-null float64\n",
      "V8        284807 non-null float64\n",
      "V9        284807 non-null float64\n",
      "V10       284807 non-null float64\n",
      "V11       284807 non-null float64\n",
      "V12       284807 non-null float64\n",
      "V13       284807 non-null float64\n",
      "V14       284807 non-null float64\n",
      "V15       284807 non-null float64\n",
      "V16       284807 non-null float64\n",
      "V17       284807 non-null float64\n",
      "V18       284807 non-null float64\n",
      "V19       284807 non-null float64\n",
      "V20       284807 non-null float64\n",
      "V21       284807 non-null float64\n",
      "V22       284807 non-null float64\n",
      "V23       284807 non-null float64\n",
      "V24       284807 non-null float64\n",
      "V25       284807 non-null float64\n",
      "V26       284807 non-null float64\n",
      "V27       284807 non-null float64\n",
      "V28       284807 non-null float64\n",
      "Amount    284807 non-null float64\n",
      "Class     284807 non-null int64\n",
      "dtypes: float64(30), int64(1)\n",
      "memory usage: 67.4 MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "      <th>V9</th>\n",
       "      <th>...</th>\n",
       "      <th>V21</th>\n",
       "      <th>V22</th>\n",
       "      <th>V23</th>\n",
       "      <th>V24</th>\n",
       "      <th>V25</th>\n",
       "      <th>V26</th>\n",
       "      <th>V27</th>\n",
       "      <th>V28</th>\n",
       "      <th>Amount</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>284807.000000</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>2.848070e+05</td>\n",
       "      <td>284807.000000</td>\n",
       "      <td>284807.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>94813.859575</td>\n",
       "      <td>3.919560e-15</td>\n",
       "      <td>5.688174e-16</td>\n",
       "      <td>-8.769071e-15</td>\n",
       "      <td>2.782312e-15</td>\n",
       "      <td>-1.552563e-15</td>\n",
       "      <td>2.010663e-15</td>\n",
       "      <td>-1.694249e-15</td>\n",
       "      <td>-1.927028e-16</td>\n",
       "      <td>-3.137024e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>1.537294e-16</td>\n",
       "      <td>7.959909e-16</td>\n",
       "      <td>5.367590e-16</td>\n",
       "      <td>4.458112e-15</td>\n",
       "      <td>1.453003e-15</td>\n",
       "      <td>1.699104e-15</td>\n",
       "      <td>-3.660161e-16</td>\n",
       "      <td>-1.206049e-16</td>\n",
       "      <td>88.349619</td>\n",
       "      <td>0.001727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>47488.145955</td>\n",
       "      <td>1.958696e+00</td>\n",
       "      <td>1.651309e+00</td>\n",
       "      <td>1.516255e+00</td>\n",
       "      <td>1.415869e+00</td>\n",
       "      <td>1.380247e+00</td>\n",
       "      <td>1.332271e+00</td>\n",
       "      <td>1.237094e+00</td>\n",
       "      <td>1.194353e+00</td>\n",
       "      <td>1.098632e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>7.345240e-01</td>\n",
       "      <td>7.257016e-01</td>\n",
       "      <td>6.244603e-01</td>\n",
       "      <td>6.056471e-01</td>\n",
       "      <td>5.212781e-01</td>\n",
       "      <td>4.822270e-01</td>\n",
       "      <td>4.036325e-01</td>\n",
       "      <td>3.300833e-01</td>\n",
       "      <td>250.120109</td>\n",
       "      <td>0.041527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-5.640751e+01</td>\n",
       "      <td>-7.271573e+01</td>\n",
       "      <td>-4.832559e+01</td>\n",
       "      <td>-5.683171e+00</td>\n",
       "      <td>-1.137433e+02</td>\n",
       "      <td>-2.616051e+01</td>\n",
       "      <td>-4.355724e+01</td>\n",
       "      <td>-7.321672e+01</td>\n",
       "      <td>-1.343407e+01</td>\n",
       "      <td>...</td>\n",
       "      <td>-3.483038e+01</td>\n",
       "      <td>-1.093314e+01</td>\n",
       "      <td>-4.480774e+01</td>\n",
       "      <td>-2.836627e+00</td>\n",
       "      <td>-1.029540e+01</td>\n",
       "      <td>-2.604551e+00</td>\n",
       "      <td>-2.256568e+01</td>\n",
       "      <td>-1.543008e+01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>54201.500000</td>\n",
       "      <td>-9.203734e-01</td>\n",
       "      <td>-5.985499e-01</td>\n",
       "      <td>-8.903648e-01</td>\n",
       "      <td>-8.486401e-01</td>\n",
       "      <td>-6.915971e-01</td>\n",
       "      <td>-7.682956e-01</td>\n",
       "      <td>-5.540759e-01</td>\n",
       "      <td>-2.086297e-01</td>\n",
       "      <td>-6.430976e-01</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.283949e-01</td>\n",
       "      <td>-5.423504e-01</td>\n",
       "      <td>-1.618463e-01</td>\n",
       "      <td>-3.545861e-01</td>\n",
       "      <td>-3.171451e-01</td>\n",
       "      <td>-3.269839e-01</td>\n",
       "      <td>-7.083953e-02</td>\n",
       "      <td>-5.295979e-02</td>\n",
       "      <td>5.600000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>84692.000000</td>\n",
       "      <td>1.810880e-02</td>\n",
       "      <td>6.548556e-02</td>\n",
       "      <td>1.798463e-01</td>\n",
       "      <td>-1.984653e-02</td>\n",
       "      <td>-5.433583e-02</td>\n",
       "      <td>-2.741871e-01</td>\n",
       "      <td>4.010308e-02</td>\n",
       "      <td>2.235804e-02</td>\n",
       "      <td>-5.142873e-02</td>\n",
       "      <td>...</td>\n",
       "      <td>-2.945017e-02</td>\n",
       "      <td>6.781943e-03</td>\n",
       "      <td>-1.119293e-02</td>\n",
       "      <td>4.097606e-02</td>\n",
       "      <td>1.659350e-02</td>\n",
       "      <td>-5.213911e-02</td>\n",
       "      <td>1.342146e-03</td>\n",
       "      <td>1.124383e-02</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>139320.500000</td>\n",
       "      <td>1.315642e+00</td>\n",
       "      <td>8.037239e-01</td>\n",
       "      <td>1.027196e+00</td>\n",
       "      <td>7.433413e-01</td>\n",
       "      <td>6.119264e-01</td>\n",
       "      <td>3.985649e-01</td>\n",
       "      <td>5.704361e-01</td>\n",
       "      <td>3.273459e-01</td>\n",
       "      <td>5.971390e-01</td>\n",
       "      <td>...</td>\n",
       "      <td>1.863772e-01</td>\n",
       "      <td>5.285536e-01</td>\n",
       "      <td>1.476421e-01</td>\n",
       "      <td>4.395266e-01</td>\n",
       "      <td>3.507156e-01</td>\n",
       "      <td>2.409522e-01</td>\n",
       "      <td>9.104512e-02</td>\n",
       "      <td>7.827995e-02</td>\n",
       "      <td>77.165000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>172792.000000</td>\n",
       "      <td>2.454930e+00</td>\n",
       "      <td>2.205773e+01</td>\n",
       "      <td>9.382558e+00</td>\n",
       "      <td>1.687534e+01</td>\n",
       "      <td>3.480167e+01</td>\n",
       "      <td>7.330163e+01</td>\n",
       "      <td>1.205895e+02</td>\n",
       "      <td>2.000721e+01</td>\n",
       "      <td>1.559499e+01</td>\n",
       "      <td>...</td>\n",
       "      <td>2.720284e+01</td>\n",
       "      <td>1.050309e+01</td>\n",
       "      <td>2.252841e+01</td>\n",
       "      <td>4.584549e+00</td>\n",
       "      <td>7.519589e+00</td>\n",
       "      <td>3.517346e+00</td>\n",
       "      <td>3.161220e+01</td>\n",
       "      <td>3.384781e+01</td>\n",
       "      <td>25691.160000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 31 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                Time            V1            V2            V3            V4  \\\n",
       "count  284807.000000  2.848070e+05  2.848070e+05  2.848070e+05  2.848070e+05   \n",
       "mean    94813.859575  3.919560e-15  5.688174e-16 -8.769071e-15  2.782312e-15   \n",
       "std     47488.145955  1.958696e+00  1.651309e+00  1.516255e+00  1.415869e+00   \n",
       "min         0.000000 -5.640751e+01 -7.271573e+01 -4.832559e+01 -5.683171e+00   \n",
       "25%     54201.500000 -9.203734e-01 -5.985499e-01 -8.903648e-01 -8.486401e-01   \n",
       "50%     84692.000000  1.810880e-02  6.548556e-02  1.798463e-01 -1.984653e-02   \n",
       "75%    139320.500000  1.315642e+00  8.037239e-01  1.027196e+00  7.433413e-01   \n",
       "max    172792.000000  2.454930e+00  2.205773e+01  9.382558e+00  1.687534e+01   \n",
       "\n",
       "                 V5            V6            V7            V8            V9  \\\n",
       "count  2.848070e+05  2.848070e+05  2.848070e+05  2.848070e+05  2.848070e+05   \n",
       "mean  -1.552563e-15  2.010663e-15 -1.694249e-15 -1.927028e-16 -3.137024e-15   \n",
       "std    1.380247e+00  1.332271e+00  1.237094e+00  1.194353e+00  1.098632e+00   \n",
       "min   -1.137433e+02 -2.616051e+01 -4.355724e+01 -7.321672e+01 -1.343407e+01   \n",
       "25%   -6.915971e-01 -7.682956e-01 -5.540759e-01 -2.086297e-01 -6.430976e-01   \n",
       "50%   -5.433583e-02 -2.741871e-01  4.010308e-02  2.235804e-02 -5.142873e-02   \n",
       "75%    6.119264e-01  3.985649e-01  5.704361e-01  3.273459e-01  5.971390e-01   \n",
       "max    3.480167e+01  7.330163e+01  1.205895e+02  2.000721e+01  1.559499e+01   \n",
       "\n",
       "       ...           V21           V22           V23           V24  \\\n",
       "count  ...  2.848070e+05  2.848070e+05  2.848070e+05  2.848070e+05   \n",
       "mean   ...  1.537294e-16  7.959909e-16  5.367590e-16  4.458112e-15   \n",
       "std    ...  7.345240e-01  7.257016e-01  6.244603e-01  6.056471e-01   \n",
       "min    ... -3.483038e+01 -1.093314e+01 -4.480774e+01 -2.836627e+00   \n",
       "25%    ... -2.283949e-01 -5.423504e-01 -1.618463e-01 -3.545861e-01   \n",
       "50%    ... -2.945017e-02  6.781943e-03 -1.119293e-02  4.097606e-02   \n",
       "75%    ...  1.863772e-01  5.285536e-01  1.476421e-01  4.395266e-01   \n",
       "max    ...  2.720284e+01  1.050309e+01  2.252841e+01  4.584549e+00   \n",
       "\n",
       "                V25           V26           V27           V28         Amount  \\\n",
       "count  2.848070e+05  2.848070e+05  2.848070e+05  2.848070e+05  284807.000000   \n",
       "mean   1.453003e-15  1.699104e-15 -3.660161e-16 -1.206049e-16      88.349619   \n",
       "std    5.212781e-01  4.822270e-01  4.036325e-01  3.300833e-01     250.120109   \n",
       "min   -1.029540e+01 -2.604551e+00 -2.256568e+01 -1.543008e+01       0.000000   \n",
       "25%   -3.171451e-01 -3.269839e-01 -7.083953e-02 -5.295979e-02       5.600000   \n",
       "50%    1.659350e-02 -5.213911e-02  1.342146e-03  1.124383e-02      22.000000   \n",
       "75%    3.507156e-01  2.409522e-01  9.104512e-02  7.827995e-02      77.165000   \n",
       "max    7.519589e+00  3.517346e+00  3.161220e+01  3.384781e+01   25691.160000   \n",
       "\n",
       "               Class  \n",
       "count  284807.000000  \n",
       "mean        0.001727  \n",
       "std         0.041527  \n",
       "min         0.000000  \n",
       "25%         0.000000  \n",
       "50%         0.000000  \n",
       "75%         0.000000  \n",
       "max         1.000000  \n",
       "\n",
       "[8 rows x 31 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'No Fraud (0) vs. Fraud (1)')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAEaCAYAAAAotpG7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcUUlEQVR4nO3df/xlVV3v8ddyUMBSfjiJM0CCOV4dqUthQJGlojiSOnjTT4DASNKk6c3SKPRygwQNI0Us5Tb8uMwYhp+rEmjYgFiZCYpilIA/0DDG7wiOMwL+AoF9/9jri4fvnPn+mPmu84XvvJ6Px3l8z1l77b3X3hzmfdba6+xTuq5DkqRWHjHXDZAkzW8GjSSpKYNGktSUQSNJasqgkSQ1ZdBIkpoyaCRJTRk00gyVUk4vpXxxGvUOKqV8o5Sy8wy2vUsp5fZSyn7b1sr5q5RyRinlC9Ood0gp5dZSyk4z2PaSUsq3SikLt62VGmTQaJuUUi4spXSllLOGLOtKKcds4/ZPrduZ+DhpW7Y7Iu8Ezui67gfjBaWUxaWUD5RS7iql3FFKed/gP2pd191R1/uLOWjvZmqoDjv/fzjXbZuGs4C3dl33Q4BSyk/W9+v1pZR7hoVV13VfAT4MnDraps5vBo1mww+A15RSntJo+7cAiyY8/nJYxVLKI0opCxq1Y9pKKb8E7A+sGShbAFwO7AUcCiwDlgKXlFLKwOr/Fzi0lPK00bV4Ujez+fl/z7CKpZQFD5Hz/0zg6cDfDBTvAPyQvu0fnGT184DjSym7tGvh9sWg0Wz4FPA54MzJKpVSFpVSLi6lfKeU8oNSyj+VUp4xje3f13XdNyc8vle3eXop5YullKNKKV8C7gGWlFJ+sZSytg6D3FVK+Uwp5bAJ7Vk3sWdUP/F+bOD1zqWUVbX3sbGU8m7gUdNo88uBf6w9lHHPB/478PKu6z7Tdd3VwArgV4BDxit1Xbce+EzdxlCllLeVUm4cUn5uKeWT9fmupZTVpZTbSil3l1L+q5Qy6X+jLRh2/r9f93FGKeULpZRjSilfBu4G9q3DhlcMnP9Pl1IOndDWb07sGZVS/qaU8g8Drx9dSjmvlHJnPf/vAh45jTa/HLiq67q7xgu6rvtO13Wv6rrur4Gvb2nFrus+BXwXeMk09qNpMGg0W/4AeFEp5dnDFtZP7H8HPBV4IXAgcBtw5SyMh+8NrASOpf8Uuw54DHAR8GvAAcBVwIdLKU+e4bb/HFhet/3L9EH2qmms92v0YTHoEOArXdd9dbyg67rrgfX0YTPo08DQc1mtBp5WSvnF8YJ6LeJldRnAW4GfA14MPAU4CvjSNNo+U/sCxwPHAPvRH89jgPcCvwo8A/hn4O9LKfvOcNtvB34dOJofh/FvT2O9Yed/JqY6/5qBHea6AZofuq67ppTyfuAdpZQDuq67f0KV59CHy9O7rrsRoJRyHP2w2O8Cb55k808qpXx3Qtnjuq67uz7fCTim67pvDCz/+IT6byylvBj4DeBt0zmmUspj6QPsVV3XXVaL/6CU8ixgqgv8+wLfmFC2CPjmkLq31WWD1tEHw1Bd191YSvkccBxwbS1+MbAjkPX1E4HPdV336fr668C/TtHuYZ4y5Pzv0nXdffX5jvTnf/3A8o9NqP9HpZTlwP+gD48plVJ2BU4Afqvruo/UstfRB0CZbF2Gn/+ZWAf87DasrwH2aDSbTqLvsbxiyLKnA98eDxmAGhSfrssmcyv99Y7Bxz0Dy8cmhAyllMeXUs4ppXypDtV9t7btiTM4nifTD5N9akL5dP6x3on+esB0TbyN+g+ZOszWAEeWUsaHko4FLh0Yrns3cFQp5d9LKe8spSybcC1ouv6TCed/IGQAbp0QMpRSnlBK+et6/u+o5//JzOz8P4X+w/AD57/rbzc/6fkvpTyC/r/bTM7/RNM5/5omezSaNV3Xfb30s89OL6XksCpDysoWygf9qOu6mydZ/r0hZe8FngCcSP8P5Q+AD/Dg6yv3s/kn48Hx//FlW/NbGhuA3SeUDRsiA9iDzXs6uwPfmmIf76OfnXZ4KeVT9JMLlo8v7Lru8lLKT9NfG3pWrf/5UsphE4JiKltz/i+iP4Y30PekfgBcwgjOf9d195dSNrL5+Z+J6Zx/TZM9Gs22P6N/X/3xhPIbgIWllKXjBaWUHemH025o0I5fBf6q67rLuq77D/rhqX0m1LkdWDzQnkL/iX3cV4AfMXChvvrlaez/Ojbvqf0r/USFJw3s8+foh80+OaHuzwKfnWwHXddtAD5KP3x2FH24rZ1Q59td172v67qV9ENrz6HvKTRTz+MzgXd1XfeRev6/xea9manO/5eBe5m98z8TU55/TZ9Bo1lVZ/n8b/pPsoM+Tn9x9n2l/yLdfvRDPzsB5zRoypeAY0op+5VSfh64mM3f7x8Dji6lPK+U8lTgbPqpx+PHcidwLvDWUsoLSyn/rZTydvohoKlcTn9BetBa4Hrgojor7mDgQvqQeWA4qA79PBP4+2nsZzX95IpXAe8b7KmUUv6slHJEKeUppZ96fjRwF/1QJKWU15VpfPFxpurw1peBY0spTy+l/AL9+Z/oY/T/jQ6t5/+v6Huh49vZBFwAnFFKObyU8tRSyjvZ/APDMMPOP7U9+wOPB3YqpexfHzsM1NmNfhLFdM6/psGgUQvn0/cGHlD/8TkC+CL9/8DX0v+j8rz6yXy2raC/SH0t8CHgMvpPuYPeCvwD/cXzf6b/1H3JhDon1va+j/560k8A/2ca+38vsGcp5cDxghoCvw6M0QfvFfSB+JLuwT91e2ht+wemsZ+P0E/FfRoD39mp7gbeAnye/jwsBZZ1XTd+Yf+n6K9btXAs/bn6LP1xfBD49wl1TqcPmw8C/0R/Xj48oc7r6QP6YuBq+qG3c6ex/9XAPjXkBl1Ffz6OB36mPv88MDjz8Ujgxq7rrpnGfjQNxZ9yltoopfwp/Sy7l85wvSuAtV3XTWt2loYrpbwF+Jmu646cwToLgBuBP+q67tJmjdvO2KOR2nkbcH2Z4b3OgE8A72rWqu3HW4EbygzudQb8NHCOITO77NFIkpqyRyNJasrv0QxnN0+Sts5mXwo2aLZgbGxsrpsgSQ8rixcvHlru0JkkqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSnvDNDI+hNPmOsm6CFo0ZnnzXUTpJGzRyNJasqgkSQ1ZdBIkpoyaCRJTRk0kqSmDBpJUlMGjSSpKYNGktSUQSNJasqgkSQ1ZdBIkpoyaCRJTRk0kqSmDBpJUlMGjSSpKYNGktSUQSNJasqgkSQ1ZdBIkpoyaCRJTRk0kqSmdhjFTiJib2AN8ATgfmBVZp4dEacCvw18q1Z9U2ZeXtd5I/BK4D7g9zJzbS1fBpwNLADOy8wzavm+wMXA7sB1wLGZeU9E7Fj3fQDwbeA3M/OW5gctSQJG16O5F3hDZj4NOBh4TUQsrcvOysz962M8ZJYCRwJPB5YB74mIBRGxAHg38AJgKXDUwHbeVre1BNhEH1LUv5sy88nAWbWeJGlERhI0mbk+M6+rz+8CbgL2nGSV5cDFmXl3Zv4ncDNwYH3cnJlfy8x76HswyyOiAM8BPlDXXw0cMbCt1fX5B4BDa31J0giMZOhsUETsA/w88GngEOC1EXEc8Fn6Xs8m+hC6ZmC1dfw4mG6dUH4Q8DjgO5l575D6e46vk5n3RsQdtf6GCe1aCays9Vi4cOE2Hef6bVpb89W2vq+kh6ORBk1E/CTwQeD3M/POiDgHOA3o6t+3A78FDOtxdAzvgXWT1GeKZQ/IzFXAqvHlGzZsmFhF2ma+rzSfLV68eGj5yGadRcQj6UPmosz8EEBm3paZ92Xm/cC59ENj0PdI9h5YfS9gbJLyDcCuEbHDhPIHbasu3wXYOLtHJ0nakpEETb0mcj5wU2a+Y6B80UC1lwBfqM8vA46MiB3rbLIlwGeAa4ElEbFvRDyKfsLAZZnZAf8IvLSuvwK4dGBbK+rzlwIfr/UlSSMwqqGzQ4Bjgf+IiH+rZW+inzW2P/1Q1i3A7wBk5g0RkcCN9DPWXpOZ9wFExGuBtfTTmy/IzBvq9v4YuDgiTgc+Tx9s1L/vjYib6XsyR7Y8UEnSg5Wu88P9EN3Y2NjUtSax/sQTZqkpmk8WnXneXDdBaqZeo9nsurh3BpAkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU3tMIqdRMTewBrgCcD9wKrMPDsidgfeD+wD3AJEZm6KiAKcDRwOfB94RWZeV7e1Aji5bvr0zFxdyw8ALgR2Bi4HXpeZ3Zb20fiQJUnVqHo09wJvyMynAQcDr4mIpcBJwFWZuQS4qr4GeAGwpD5WAucA1NA4BTgIOBA4JSJ2q+ucU+uOr7eslm9pH5KkERhJ0GTm+vEeSWbeBdwE7AksB1bXaquBI+rz5cCazOwy8xpg14hYBDwfuDIzN9ZeyZXAsrrssZl5dWZ29L2nwW0N24ckaQRGMnQ2KCL2AX4e+DSwR2auhz6MIuLxtdqewK0Dq62rZZOVrxtSziT7mNiulfQ9IjKThQsXbu0hArB+m9bWfLWt7yvp4WikQRMRPwl8EPj9zLwzIrZUtQwp67aifNoycxWwanzdDRs2zGR1aVp8X2k+W7x48dDykc06i4hH0ofMRZn5oVp8Wx32ov69vZavA/YeWH0vYGyK8r2GlE+2D0nSCIwkaOossvOBmzLzHQOLLgNW1OcrgEsHyo+LiBIRBwN31OGvtcBhEbFbnQRwGLC2LrsrIg6u+zpuwraG7UOSNAKjGjo7BDgW+I+I+Lda9ibgDCAj4pXAfwEvq8sup5/afDP99ObjATJzY0ScBlxb6705MzfW56/mx9ObP1ofTLIPSdIIlK6b0aWM7UU3NjY2da1JrD/xhFlqiuaTRWeeN9dNkJqp12g2u2bunQEkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU1NO2gi4g+3UP762WuOJGm+mUmP5k+2UH7ybDREkjQ/7TBVhYh4Tn26ICKezYN/D/pJwF0tGiZJmh+mDBrg/Pp3J+CCgfIO+CbwP2e7UZKk+WPKoMnMfQEiYk1mHte+SZKk+WQ6PRoABkMmIh4xYdn9s9koSdL8Me2giYhfAN4N/Bz9MBr012s6YMHsN02SNB9MO2iA1cCHgd8Cvt+mOZKk+WYmQfNE4H9lZteqMZKk+Wcm36O5BDisVUMkSfPTTHo0OwGXRMQn6ac1P2Cq2WgRcQHwQuD2zNyvlp0K/DbwrVrtTZl5eV32RuCVwH3A72Xm2lq+DDib/prQeZl5Ri3fF7gY2B24Djg2M++JiB2BNcABwLeB38zMW2ZwzJKkbTSTHs2NwNuAfwW+OuExlQuBZUPKz8rM/etjPGSWAkcCT6/rvCciFkTEAvrJCC8AlgJH1brUdp2VmUuATfQhRf27KTOfDJxV60mSRmgm05v/dGt3kpmfiIh9pll9OXBxZt4N/GdE3AwcWJfdnJlfA4iIi4HlEXET8Bzg6FpnNXAqcE7d1qm1/APAX0VE8TqTJI3OTKY3P2dLyzLz41u5/9dGxHHAZ4E3ZOYmYE/gmoE662oZwK0Tyg8CHgd8JzPvHVJ/z/F1MvPeiLij1t+wle2VJM3QTK7RnD/h9U8Bj6L/h/1JW7Hvc4DT6L+Hcxrwdvqp02VI3Y7hw3zdJPWZYtmDRMRKYCVAZrJw4cLJ2j6l9du0tuarbX1fSQ9HMxk623fwdb1mcjJbeVPNzLxtYFvnAh+pL9cBew9U3QsYq8+HlW8Ado2IHWqvZrD++LbWRcQOwC7Axi20ZxWwqr7sNmyw06PZ5/tK89nixYuHlm/1D59l5n3AW4A/2pr1I2LRwMuXAF+ozy8DjoyIHetssiXAZ4BrgSURsW9EPIp+wsBl9XrLPwIvreuvAC4d2NaK+vylwMe9PiNJozWTobNhngdMeZ+ziPhb4FnAwohYB5wCPCsi9qcfyroF+B2AzLwhIpJ+ltu9wGtqqBERrwXW0k9vviAzb6i7+GPg4og4Hfg8Px7mOx94b51QsJE+nCRJI1S6bnof8CPiVh58fePR9N+t+d3MXNOgbXOpGxsbm7rWJNafeMIsNUXzyaIzz5vrJkjN1KGzza6Nz6RHc8yE198DvpyZd25DuyRJ89xMJgP8MzzwEwF7ALf58wCSpKnM5Hs0j6H/Zv5vAo8EflS/NPl7mXlHo/ZJkh7mZjLr7C+BnwB+Fti5/n008K4G7ZIkzRMzuUazDHhSZo7/Fs2XI+J4pnevM0nSdmomPZof0t8NYNBC4O7Za44kab6ZSY/mPODKiHgH8HX6H0L7A+DcFg2TJM0PMwmatwDfAF4OLKa/zcufZ+bEe6BJkvSAmQydnQ18KTOfm5lLM/O5wE0R8c5GbZMkzQMzCZqj6G/nP+hz/Ph3YCRJ2sxMgqajv8fYoAUz3IYkaTszk5D4F+C0emeA8TsEnFrLJUkaaiaTAV5H/5sx6yPi68BP0/++14taNEySND9Mu0eTmeuAXwCWA2cCRwAH1HJJkoaa0e/R1JtoXlMfkiRNyQv5kqSmDBpJUlMGjSSpKYNGktSUQSNJasqgkSQ1ZdBIkpoyaCRJTRk0kqSmDBpJUlMGjSSpKYNGktSUQSNJampGd2/eWhFxAfBC4PbM3K+W7Q68H9gHuAWIzNwUEQU4Gzgc+D7wisy8rq6zAji5bvb0zFxdyw8ALgR2Bi4HXpeZ3Zb20fhwJUkDRtWjuRBYNqHsJOCqzFwCXFVfA7wAWFIfK4Fz4IFgOgU4CDgQOCUidqvrnFPrjq+3bIp9SJJGZCRBk5mfADZOKF4OrK7PV9P/kNp4+ZrM7DLzGmDXiFgEPB+4MjM31l7JlcCyuuyxmXl1ZnbAmgnbGrYPSdKIjGTobAv2yMz1AJm5PiIeX8v3BG4dqLeulk1Wvm5I+WT72ExErKTvFZGZLFy4cGuPC+h/41qaaFvfV9LD0VwGzZaUIWXdVpTPSGauAlaNr79hw4aZbkKaku8rzWeLFy8eWj6Xs85uq8Ne1L+31/J1wN4D9fYCxqYo32tI+WT7kCSNyFwGzWXAivp8BXDpQPlxEVEi4mDgjjr8tRY4LCJ2q5MADgPW1mV3RcTBdcbacRO2NWwfkqQRGdX05r8FngUsjIh19LPHzgAyIl4J/Bfwslr9cvqpzTfTT28+HiAzN0bEacC1td6bM3N8gsGr+fH05o/WB5PsQ5I0IqXrZnw5Y3vQjY2NTV1rEutPPGGWmqL5ZNGZ5811E6Rm6jWaza6be2cASVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJTBo0kqSmDRpLU1A5z3YCIuAW4C7gPuDcznxERuwPvB/YBbgEiMzdFRAHOBg4Hvg+8IjOvq9tZAZxcN3t6Zq6u5QcAFwI7A5cDr8vMbiQHJ0l6yPRonp2Z+2fmM+rrk4CrMnMJcFV9DfACYEl9rATOAajBdApwEHAgcEpE7FbXOafWHV9vWfvDkSSNe6gEzUTLgdX1+WrgiIHyNZnZZeY1wK4RsQh4PnBlZm7MzE3AlcCyuuyxmXl17cWsGdiWJGkE5nzoDOiAKyKiA/46M1cBe2TmeoDMXB8Rj6919wRuHVh3XS2brHzdkPLNRMRK+p4PmcnChQu36aDWb9Pamq+29X0lPRw9FILmkMwcq2FyZUR8cZK6ZUhZtxXlm6kBt2q8zoYNGyZphrR1fF9pPlu8ePHQ8jkfOsvMsfr3duAS+msst9VhL+rf22v1dcDeA6vvBYxNUb7XkHJJ0ojMadBExE9ExGPGnwOHAV8ALgNW1GorgEvr88uA4yKiRMTBwB11iG0tcFhE7FYnARwGrK3L7oqIg+uMteMGtiVJGoG57tHsAXwyIq4HPgP8fWb+A3AG8LyI+ArwvPoa+unJXwNuBs4FfhcgMzcCpwHX1sebaxnAq4Hz6jpfBT46guOSJFWl6/xKyRDd2Ni2jbCtP/GEWWqK5pNFZ543102QmqnXaDa7Nj7XPRpJ0jxn0EiSmjJoJElNGTSSpKYMGklSUwaNJKkpg0aS1JRBI0lqyqCRJDVl0EiSmjJoJElNGTSSpKYMGklSUwaNJKkpg0aS1JRBI0lqyqCRJDVl0EiSmjJoJElNGTSSpKYMGklSUwaNJKkpg0aS1JRBI0lqyqCRJDVl0EiSmjJoJElNGTSSpKZ2mOsGjEJELAPOBhYA52XmGXPcJEnabsz7Hk1ELADeDbwAWAocFRFL57ZVkrT92B56NAcCN2fm1wAi4mJgOXDjnLZKmiOvWH31XDdBD0EXrvilZtveHoJmT+DWgdfrgIMmVoqIlcBKgMxk8eLF27TTxRddvk3rS61c8cbfmOsmaDuzPQRNGVLWTSzIzFXAqvbN2f5ExGcz8xlz3Q5pIt+bozHvr9HQ92D2Hni9FzA2R22RpO3O9tCjuRZYEhH7At8AjgSOntsmSdL2Y973aDLzXuC1wFrgpr4ob5jbVm13HJLUQ5XvzREoXbfZ5QpJkmbNvO/RSJLmlkEjSWpqe5gMoDnirX/0UBURFwAvBG7PzP3muj3znT0aNeGtf/QQdyGwbK4bsb0waNTKA7f+ycx7gPFb/0hzLjM/AWyc63ZsLwwatTLs1j97zlFbJM0hg0atTOvWP5LmP4NGrXjrH0mAs87Ujrf+kQR4ZwA1FBGHA++kn958QWa+ZY6bJAEQEX8LPAtYCNwGnJKZ589po+Yxg0aS1JTXaCRJTRk0kqSmDBpJUlMGjSSpKYNGktSUQSM9xETEqRHxN3PdDmm2+IVNaY5ExNHA64GnAncB/wb4XSPNOwaNNAci4vXAScCrgLXAPfS3rV8OfG8OmybNOoNGGrGI2AV4M3B8Zn5oYNGHgQ9HxKkT6v8/4JnAzsD1wKsz84a67HDgL+jvK3cncFZm/kVELKT/zZVfAe4HbgB+LTPvb3ho0lBeo5FG75eAnYBLpln/o8AS4PHAdcBFA8vOB34nMx8D7Ad8vJa/gf7Gpj8F7AG8Ce+erTlij0YavccBGzLz3ulUzswLxp/X3s6miNglM+8AfgQsjYjrM3MTsKlW/RGwCHhiZt4M/MtsHoA0EwaNNHrfBhZGxA5ThU39Sey3AC+j752MD30tBO4AfgM4GTgjIv4dOCkzrwbOBE4FrogIgFWZeUaDY5Gm5NCZNHpXAz8EjphG3aPpJwg8F9gF2KeWF4DMvDYzl9MPq/0dkLX8rsx8Q2Y+CXgR8PqIOHQ2D0KaLns00ohl5h0R8SfAuyPiXuAK+qGu5wLPBr4/UP0xwN30vaBHA28dXxARj6Lv6XykbvNO4L667IXAF4Gv0k8SuG98mTRq9mikOZCZ76D/Ds3JwLeAW4HX0vdKBq0Bvk7/43E3AtdMWH4scEsNmVcBx9TyJcDHgO/S96Dek5n/NOsHIk2Dv0cjSWrKHo0kqSmDRpLUlEEjSWrKoJEkNWXQSJKaMmgkSU0ZNJKkpgwaSVJT/x9fgzh3m4SvVAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# taking a closer look at the class variable\n",
    "sns.countplot('Class', data = df)\n",
    "plt.title('No Fraud (0) vs. Fraud (1)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = df.drop(['Class'], axis=1)\n",
    "y = df['Class']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 284315), (1, 284315)]\n"
     ]
    }
   ],
   "source": [
    "# imports \n",
    "from imblearn.over_sampling import RandomOverSampler\n",
    "\n",
    "# random oversampling\n",
    "ros = RandomOverSampler(random_state=0)\n",
    "X_resampled, y_resampled = ros.fit_resample(x, y)\n",
    "\n",
    "# using Counter to display results of naive oversampling\n",
    "from collections import Counter\n",
    "print(sorted(Counter(y_resampled).items()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_resampled[0:50,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "284315"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_resampled.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 1, 1, 1], dtype=int64)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_resampled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 284315), (1, 284315)]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# importing SMOTE\n",
    "from imblearn.over_sampling import SMOTE\n",
    "\n",
    "# applying SMOTE to our data and checking the class counts\n",
    "X_resampled2, y_resampled2 = SMOTE().fit_resample(x, y)\n",
    "print(sorted(Counter(y_resampled2).items()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 284315), (1, 284298)]\n"
     ]
    }
   ],
   "source": [
    "# importing ADASYN\n",
    "from imblearn.over_sampling import ADASYN\n",
    "\n",
    "# applying ADASYN\n",
    "X_resampled3, y_resampled3 = ADASYN().fit_resample(x, y)\n",
    "print(sorted(Counter(y_resampled3).items()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 284315), (1, 284315)]\n"
     ]
    }
   ],
   "source": [
    "# BorderlineSMOTE\n",
    "from imblearn.over_sampling import BorderlineSMOTE\n",
    "\n",
    "X_resampled4, y_resampled4 = BorderlineSMOTE().fit_resample(x, y)\n",
    "print(sorted(Counter(y_resampled4).items()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model_train(X, Y):\n",
    "    # Define search parameter values\n",
    "    depth = [5,10,15,20]\n",
    "    tree_amt = [50,100,150]\n",
    "    RFC = RandomForestClassifier(max_depth=5, random_state=42, n_estimators=50)\n",
    "\n",
    "    # Create a parameter grid: map the parameter names to search values\n",
    "    param_grid = dict(max_depth=depth, n_estimators=tree_amt)\n",
    "    print(param_grid)\n",
    "\n",
    "    # Instantiate and fit the grid\n",
    "    grid = GridSearchCV(RFC, param_grid, cv=10, scoring='accuracy')\n",
    "    grid.fit(X,Y)\n",
    "    \n",
    "    return grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'max_depth': [5, 10, 15, 20], 'n_estimators': [50, 100, 150]}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_validation.py:530: FutureWarning: From version 0.22, errors during fit will result in a cross validation score of NaN by default. Use error_score='raise' if you want an exception raised or error_score=np.nan to adopt the behavior from version 0.22.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "ename": "MemoryError",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mMemoryError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-33-81b62ff6f72a>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mnaive_sampling_model\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mmodel_train\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_resampled\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my_resampled\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m<ipython-input-32-fa6825037f87>\u001b[0m in \u001b[0;36mmodel_train\u001b[1;34m(X, Y)\u001b[0m\n\u001b[0;32m     13\u001b[0m     \u001b[1;31m# Also called exhaustive grid search, as it tries all permutations of n_neighbors/weight_options\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     14\u001b[0m     \u001b[1;31m# 30 neighbors x 2 weight options x 10 fold cross validation = 600 models trained and tested\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 15\u001b[1;33m     \u001b[0mgrid\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mY\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     16\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     17\u001b[0m     \u001b[1;32mreturn\u001b[0m \u001b[0mgrid\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, groups, **fit_params)\u001b[0m\n\u001b[0;32m    686\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mresults\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    687\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 688\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_run_search\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    689\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    690\u001b[0m         \u001b[1;31m# For multi-metric evaluation, store the best_index_, best_params_ and\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36m_run_search\u001b[1;34m(self, evaluate_candidates)\u001b[0m\n\u001b[0;32m   1147\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m_run_search\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1148\u001b[0m         \u001b[1;34m\"\"\"Search all candidates in param_grid\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1149\u001b[1;33m         \u001b[0mevaluate_candidates\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mParameterGrid\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mparam_grid\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1150\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1151\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py\u001b[0m in \u001b[0;36mevaluate_candidates\u001b[1;34m(candidate_params)\u001b[0m\n\u001b[0;32m    665\u001b[0m                                \u001b[1;32mfor\u001b[0m \u001b[0mparameters\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mtrain\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtest\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    666\u001b[0m                                in product(candidate_params,\n\u001b[1;32m--> 667\u001b[1;33m                                           cv.split(X, y, groups)))\n\u001b[0m\u001b[0;32m    668\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    669\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mout\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m<\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, iterable)\u001b[0m\n\u001b[0;32m    922\u001b[0m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_iterating\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_original_iterator\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    923\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 924\u001b[1;33m             \u001b[1;32mwhile\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdispatch_one_batch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    925\u001b[0m                 \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    926\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36mdispatch_one_batch\u001b[1;34m(self, iterator)\u001b[0m\n\u001b[0;32m    757\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    758\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 759\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dispatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtasks\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    760\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    761\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m_dispatch\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    714\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    715\u001b[0m             \u001b[0mjob_idx\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 716\u001b[1;33m             \u001b[0mjob\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatch\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    717\u001b[0m             \u001b[1;31m# A job can complete so quickly than its callback is\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    718\u001b[0m             \u001b[1;31m# called before we get here, causing self._jobs to\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36mapply_async\u001b[1;34m(self, func, callback)\u001b[0m\n\u001b[0;32m    180\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    181\u001b[0m         \u001b[1;34m\"\"\"Schedule a func to be run\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 182\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mImmediateResult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    183\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    184\u001b[0m             \u001b[0mcallback\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    547\u001b[0m         \u001b[1;31m# Don't delay the application, to avoid keeping the input\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    548\u001b[0m         \u001b[1;31m# arguments in memory\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 549\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    550\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    551\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_validation.py\u001b[0m in \u001b[0;36m_fit_and_score\u001b[1;34m(estimator, X, y, scorer, train, test, verbose, parameters, fit_params, return_train_score, return_parameters, return_n_test_samples, return_times, return_estimator, error_score)\u001b[0m\n\u001b[0;32m    514\u001b[0m             \u001b[0mestimator\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    515\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 516\u001b[1;33m             \u001b[0mestimator\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mfit_params\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    517\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    518\u001b[0m     \u001b[1;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\ensemble\\forest.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[0;32m    328\u001b[0m                     \u001b[0mt\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msample_weight\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtrees\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    329\u001b[0m                     verbose=self.verbose, class_weight=self.class_weight)\n\u001b[1;32m--> 330\u001b[1;33m                 for i, t in enumerate(trees))\n\u001b[0m\u001b[0;32m    331\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    332\u001b[0m             \u001b[1;31m# Collect newly grown trees\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, iterable)\u001b[0m\n\u001b[0;32m    922\u001b[0m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_iterating\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_original_iterator\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    923\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 924\u001b[1;33m             \u001b[1;32mwhile\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdispatch_one_batch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    925\u001b[0m                 \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    926\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36mdispatch_one_batch\u001b[1;34m(self, iterator)\u001b[0m\n\u001b[0;32m    757\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    758\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 759\u001b[1;33m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_dispatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtasks\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    760\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    761\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m_dispatch\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    714\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    715\u001b[0m             \u001b[0mjob_idx\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 716\u001b[1;33m             \u001b[0mjob\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatch\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcb\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    717\u001b[0m             \u001b[1;31m# A job can complete so quickly than its callback is\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    718\u001b[0m             \u001b[1;31m# called before we get here, causing self._jobs to\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36mapply_async\u001b[1;34m(self, func, callback)\u001b[0m\n\u001b[0;32m    180\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mapply_async\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    181\u001b[0m         \u001b[1;34m\"\"\"Schedule a func to be run\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 182\u001b[1;33m         \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mImmediateResult\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    183\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mcallback\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    184\u001b[0m             \u001b[0mcallback\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\_parallel_backends.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, batch)\u001b[0m\n\u001b[0;32m    547\u001b[0m         \u001b[1;31m# Don't delay the application, to avoid keeping the input\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    548\u001b[0m         \u001b[1;31m# arguments in memory\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 549\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbatch\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    550\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    551\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mget\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\joblib\\parallel.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[1;34m(.0)\u001b[0m\n\u001b[0;32m    223\u001b[0m         \u001b[1;32mwith\u001b[0m \u001b[0mparallel_backend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_backend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mn_jobs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_n_jobs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    224\u001b[0m             return [func(*args, **kwargs)\n\u001b[1;32m--> 225\u001b[1;33m                     for func, args, kwargs in self.items]\n\u001b[0m\u001b[0;32m    226\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    227\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\ensemble\\forest.py\u001b[0m in \u001b[0;36m_parallel_build_trees\u001b[1;34m(tree, forest, X, y, sample_weight, tree_idx, n_trees, verbose, class_weight)\u001b[0m\n\u001b[0;32m    116\u001b[0m             \u001b[0mcurr_sample_weight\u001b[0m \u001b[1;33m*=\u001b[0m \u001b[0mcompute_sample_weight\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'balanced'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindices\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    117\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 118\u001b[1;33m         \u001b[0mtree\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msample_weight\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcurr_sample_weight\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    119\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    120\u001b[0m         \u001b[0mtree\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msample_weight\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0msample_weight\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcheck_input\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\tree\\tree.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight, check_input, X_idx_sorted)\u001b[0m\n\u001b[0;32m    814\u001b[0m             \u001b[0msample_weight\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0msample_weight\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    815\u001b[0m             \u001b[0mcheck_input\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcheck_input\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 816\u001b[1;33m             X_idx_sorted=X_idx_sorted)\n\u001b[0m\u001b[0;32m    817\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    818\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\sklearn\\tree\\tree.py\u001b[0m in \u001b[0;36mfit\u001b[1;34m(self, X, y, sample_weight, check_input, X_idx_sorted)\u001b[0m\n\u001b[0;32m    164\u001b[0m             \u001b[1;32mfor\u001b[0m \u001b[0mk\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mn_outputs_\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    165\u001b[0m                 classes_k, y_encoded[:, k] = np.unique(y[:, k],\n\u001b[1;32m--> 166\u001b[1;33m                                                        return_inverse=True)\n\u001b[0m\u001b[0;32m    167\u001b[0m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclasses_\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mclasses_k\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    168\u001b[0m                 \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mn_classes_\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mclasses_k\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\numpy\\lib\\arraysetops.py\u001b[0m in \u001b[0;36munique\u001b[1;34m(ar, return_index, return_inverse, return_counts, axis)\u001b[0m\n\u001b[0;32m    262\u001b[0m     \u001b[0mar\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0masanyarray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mar\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    263\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0maxis\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 264\u001b[1;33m         \u001b[0mret\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_unique1d\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mar\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreturn_index\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreturn_inverse\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreturn_counts\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    265\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0m_unpack_tuple\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mret\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    266\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\lib\\site-packages\\numpy\\lib\\arraysetops.py\u001b[0m in \u001b[0;36m_unique1d\u001b[1;34m(ar, return_index, return_inverse, return_counts)\u001b[0m\n\u001b[0;32m    307\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    308\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0moptional_indices\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 309\u001b[1;33m         \u001b[0mperm\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mar\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0margsort\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkind\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'mergesort'\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mreturn_index\u001b[0m \u001b[1;32melse\u001b[0m \u001b[1;34m'quicksort'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    310\u001b[0m         \u001b[0maux\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mar\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mperm\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    311\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mMemoryError\u001b[0m: "
     ]
    }
   ],
   "source": [
    "naive_sampling_model = model_train(X_resampled, y_resampled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}