05-Implicit_Feedback_with_the_triplet_loss_with_Tensorflow_Rendered.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Triplet Loss for Implicit Feedback Neural Recommender Systems\n",
    "\n",
    "Goals:\n",
    "- Understand bi-linear recommendation system only using positive feedback data\n",
    "- Use Margin Based Comparator / Triplet Loss\n",
    "- Build deep learning architecture using similar deign principle\n",
    "\n",
    "This notebook is inspired by Oliver Grisel Notebook who used Keras\n",
    "https://github.com/ogrisel for building the moels. We will be using Basic Tensorflow APIs instead. You can also look into Maciej Kula's work [Recommendations in Keras using triplet loss](\n",
    "https://github.com/maciejkula/triplet_recommendations_keras) that uses BPR ( Bayesian Personalized Ranking ). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.5.0\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os\n",
    "import os.path as op\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from tensorflow.contrib import layers\n",
    "import tensorflow as tf\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>90570.000000</td>\n",
       "      <td>90570.000000</td>\n",
       "      <td>90570.000000</td>\n",
       "      <td>9.057000e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>461.494038</td>\n",
       "      <td>428.104891</td>\n",
       "      <td>3.523827</td>\n",
       "      <td>8.835073e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>266.004364</td>\n",
       "      <td>333.088029</td>\n",
       "      <td>1.126073</td>\n",
       "      <td>5.341684e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.747247e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>256.000000</td>\n",
       "      <td>174.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>8.794484e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>442.000000</td>\n",
       "      <td>324.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.828143e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>682.000000</td>\n",
       "      <td>636.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.882049e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>943.000000</td>\n",
       "      <td>1682.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>8.932866e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id       item_id        rating     timestamp\n",
       "count  90570.000000  90570.000000  90570.000000  9.057000e+04\n",
       "mean     461.494038    428.104891      3.523827  8.835073e+08\n",
       "std      266.004364    333.088029      1.126073  5.341684e+06\n",
       "min        1.000000      1.000000      1.000000  8.747247e+08\n",
       "25%      256.000000    174.000000      3.000000  8.794484e+08\n",
       "50%      442.000000    324.000000      4.000000  8.828143e+08\n",
       "75%      682.000000    636.000000      4.000000  8.882049e+08\n",
       "max      943.000000   1682.000000      5.000000  8.932866e+08"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Base Path for MovieLens dataset\n",
    "ML_100K_PATH = os.path.join('processed','ml-100k','ml-100k')\n",
    "\n",
    "data_train = pd.read_csv(op.join(ML_100K_PATH, 'ua.base'), sep='\\t',\n",
    "                        names=[\"user_id\", \"item_id\", \"rating\", \"timestamp\"])\n",
    "data_test = pd.read_csv(op.join(ML_100K_PATH, 'ua.test'), sep='\\t',\n",
    "                        names=[\"user_id\", \"item_id\", \"rating\", \"timestamp\"])\n",
    "\n",
    "data_train.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_release_year(x):\n",
    "    splits = str(x).split('-')\n",
    "    if(len(splits) == 3):\n",
    "        return int(splits[2])\n",
    "    else:\n",
    "        return 1920\n",
    "    \n",
    "\n",
    "m_cols = ['item_id', 'title', 'release_date', 'video_release_date', 'imdb_url']\n",
    "items = pd.read_csv(op.join(ML_100K_PATH, 'u.item'), sep='|',\n",
    "                    names=m_cols, usecols=range(5), encoding='latin-1')\n",
    "items['release_year'] = items['release_date'].map(get_release_year)\n",
    "\n",
    "data_train = pd.merge(data_train, items)\n",
    "data_test = pd.merge(data_test, items)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>title</th>\n",
       "      <th>release_date</th>\n",
       "      <th>video_release_date</th>\n",
       "      <th>imdb_url</th>\n",
       "      <th>release_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>874965758</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>888550871</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>883599478</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>877888877</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>882140487</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>01-Jan-1995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
       "      <td>1995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  item_id  rating  timestamp             title release_date  \\\n",
       "0        1        1       5  874965758  Toy Story (1995)  01-Jan-1995   \n",
       "1        2        1       4  888550871  Toy Story (1995)  01-Jan-1995   \n",
       "2        6        1       4  883599478  Toy Story (1995)  01-Jan-1995   \n",
       "3       10        1       4  877888877  Toy Story (1995)  01-Jan-1995   \n",
       "4       13        1       3  882140487  Toy Story (1995)  01-Jan-1995   \n",
       "\n",
       "   video_release_date                                           imdb_url  \\\n",
       "0                 NaN  http://us.imdb.com/M/title-exact?Toy%20Story%2...   \n",
       "1                 NaN  http://us.imdb.com/M/title-exact?Toy%20Story%2...   \n",
       "2                 NaN  http://us.imdb.com/M/title-exact?Toy%20Story%2...   \n",
       "3                 NaN  http://us.imdb.com/M/title-exact?Toy%20Story%2...   \n",
       "4                 NaN  http://us.imdb.com/M/title-exact?Toy%20Story%2...   \n",
       "\n",
       "   release_year  \n",
       "0          1995  \n",
       "1          1995  \n",
       "2          1995  \n",
       "3          1995  \n",
       "4          1995  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n_users=944, n_items=1683\n"
     ]
    }
   ],
   "source": [
    "max_user_id = max(data_train['user_id'].max(), data_test['user_id'].max())\n",
    "max_item_id = max(data_train['item_id'].max(), data_test['item_id'].max())\n",
    "\n",
    "n_users = max_user_id + 1\n",
    "n_items = max_item_id + 1\n",
    "\n",
    "print('n_users=%d, n_items=%d' % (n_users, n_items))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implicit feedback data\n",
    "\n",
    "Consider ratings >= 4 as positive feed back and ignore the rest:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.5238268742409184\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAD8CAYAAAC/1zkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAFu9JREFUeJzt3X+w3XV95/HnywCKP0GJliaxod2sa3QqYBazw27HooUArcFZ2cXOSnSxqRa2OnVmjc5OsSozONNCl9VicckarBrxJ6nG0qi0TmfKj4tSIESXu5iVGMZEg4DFwgbf+8f5XDm9nOSeG77nnnvN8zFz5n7P+/v5nu/7fOHmdb8/zvekqpAkqQtPGXcDkqSfH4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTNHjLuBuXbcccfV8uXLx92GJC0ot9566w+qavFM4w67UFm+fDkTExPjbkOSFpQk/3eYcR7+kiR1xlCRJHXGUJEkdcZQkSR1xlCRJHXGUJEkdcZQkSR1xlCRJHXGUJEkdeaw+0S9pPlj+YYvjWW9Oy89eyzrPRy4pyJJ6szIQiXJ05LcnOQfkmxP8ketfkKSm5LcneRTSY5q9ae255Nt/vK+13pXq387yRl99TWtNplkw6jeiyRpOKPcU3kEOK2qXgacCKxJshr4AHB5Va0A7gcuaOMvAO6vqn8BXN7GkWQlcB7wEmAN8GdJFiVZBHwIOBNYCby+jZUkjcnIQqV6ftyeHtkeBZwGfKbVNwHntOm17Tlt/quSpNU3V9UjVfUdYBI4pT0mq+qeqnoU2NzGSpLGZKTnVNoexW3AHmAb8H+AH1XV/jZkF7CkTS8B7gVo8x8Antdfn7bMgeqD+lifZCLJxN69e7t4a5KkAUYaKlX1WFWdCCylt2fx4kHD2s8cYN5s64P6uKqqVlXVqsWLZ/yOGUnSIZqTq7+q6kfA3wCrgWOSTF3KvBTY3aZ3AcsA2vznAPv669OWOVBdkjQmo7z6a3GSY9r00cCrgR3ADcDr2rB1wHVtekt7Tpv/taqqVj+vXR12ArACuBm4BVjRriY7it7J/C2jej+SpJmN8sOPxwOb2lVaTwGuraovJrkL2Jzk/cA3gavb+KuBjyWZpLeHch5AVW1Pci1wF7AfuLCqHgNIchFwPbAI2FhV20f4fiRJMxhZqFTV7cBJA+r30Du/Mr3+T8C5B3itS4BLBtS3AlufdLOSpE74iXpJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnRhYqSZYluSHJjiTbk7yt1d+T5HtJbmuPs/qWeVeSySTfTnJGX31Nq00m2dBXPyHJTUnuTvKpJEeN6v1IkmY2yj2V/cA7qurFwGrgwiQr27zLq+rE9tgK0OadB7wEWAP8WZJFSRYBHwLOBFYCr+97nQ+011oB3A9cMML3I0mawchCparuq6pvtOmHgB3AkoMsshbYXFWPVNV3gEnglPaYrKp7qupRYDOwNkmA04DPtOU3AeeM5t1IkoYxJ+dUkiwHTgJuaqWLktyeZGOSY1ttCXBv32K7Wu1A9ecBP6qq/dPqkqQxGXmoJHkm8Fng7VX1IHAl8CvAicB9wJ9MDR2weB1CfVAP65NMJJnYu3fvLN+BJGlYIw2VJEfSC5SPV9XnAKrq+1X1WFX9FPgIvcNb0NvTWNa3+FJg90HqPwCOSXLEtPoTVNVVVbWqqlYtXry4mzcnSXqCI2YecmjaOY+rgR1VdVlf/fiquq89fS1wZ5veAnwiyWXALwIrgJvp7ZGsSHIC8D16J/N/u6oqyQ3A6+idZ1kHXDeq9yON2vINXxrbundeevbY1q2fLyMLFeBU4A3AHUlua7V307t660R6h6p2Ar8LUFXbk1wL3EXvyrELq+oxgCQXAdcDi4CNVbW9vd47gc1J3g98k16ISZLGZGShUlV/x+DzHlsPsswlwCUD6lsHLVdV9/D44TNJ0pj5iXpJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmcMFUlSZwwVSVJnDBVJUmdGFipJliW5IcmOJNuTvK3Vn5tkW5K7289jWz1JrkgymeT2JCf3vda6Nv7uJOv66i9Pckdb5ookGdX7kSTNbJR7KvuBd1TVi4HVwIVJVgIbgK9W1Qrgq+05wJnAivZYD1wJvRACLgZeAZwCXDwVRG3M+r7l1ozw/UiSZjCyUKmq+6rqG236IWAHsARYC2xqwzYB57TptcA11XMjcEyS44EzgG1Vta+q7ge2AWvavGdX1d9XVQHX9L2WJGkM5uScSpLlwEnATcALquo+6AUP8Pw2bAlwb99iu1rtYPVdA+qSpDEZeagkeSbwWeDtVfXgwYYOqNUh1Af1sD7JRJKJvXv3ztSyJOkQDRUqSV56KC+e5Eh6gfLxqvpcK3+/Hbqi/dzT6ruAZX2LLwV2z1BfOqD+BFV1VVWtqqpVixcvPpS3IkkawrB7Kh9OcnOS30tyzDALtCuxrgZ2VNVlfbO2AFNXcK0Druurn9+uAlsNPNAOj10PnJ7k2HaC/nTg+jbvoSSr27rO73stSdIYHDHMoKr6t0lWAP8ZmEhyM/C/qmrbQRY7FXgDcEeS21rt3cClwLVJLgC+C5zb5m0FzgImgYeBN7V170vyPuCWNu69VbWvTb8V+ChwNPDl9pAkjclQoQJQVXcn+W/ABHAFcFLbQ3h336Gt/vF/x+DzHgCvGjC+gAsPsO6NwMYB9QngkA7NSZK6N+w5lV9Ncjm9y4JPA36rff7kNODyEfYnSVpAht1T+SDwEXp7JT+ZKlbV7rb3IkkawvINXxrLendeevacrGfYUDkL+ElVPQaQ5CnA06rq4ar62Mi6kyQtKMNe/fUVeifDpzy91SRJ+plhQ+VpVfXjqSdt+umjaUmStFANGyr/OO2uwS8HfnKQ8ZKkw9Cw51TeDnw6ydQn1o8H/uNoWpIkLVTDfvjxliT/CngRvc+efKuq/t9IO5MkLThDf/gR+NfA8rbMSUmoqmtG0pUkaUEaKlSSfAz4FeA24LFWnvoOE0mSgOH3VFYBK9utVCRJGmjYq7/uBH5hlI1Ikha+YfdUjgPuancnfmSqWFWvGUlXkqQFadhQec8om5Ak/XwY9pLiv03yS8CKqvpKkqcDi0bbmiRpoRn21ve/A3wG+PNWWgJ8YVRNSZIWpmFP1F9I75scH4TeF3YBzx9VU5KkhWnYUHmkqh6depLkCHqfU5Ek6WeGDZW/TfJu4OgkvwF8GvjL0bUlSVqIhg2VDcBe4A7gd4GtgN/4KEn6Z4a9+uun9L5O+COjbUeStJANe++v7zDgHEpV/XLnHUmSFqzZ3PtrytOAc4Hndt+OJGkhG+qcSlX9sO/xvar6U+C0EfcmSVpghv3w48l9j1VJ3gI8a4ZlNibZk+TOvtp7knwvyW3tcVbfvHclmUzy7SRn9NXXtNpkkg199ROS3JTk7iSfSnLUrN65JKlzwx7++pO+6f3ATuA/zLDMR4EP8sTvXLm8qv64v5BkJXAe8BLgF4GvJPmXbfaHgN8AdgG3JNlSVXcBH2ivtTnJh4ELgCuHfD+SpBEY9uqvX5/tC1fV15MsH3L4WmBzVT0CfCfJJHBKmzdZVfcAJNkMrE2yg97ht99uYzbRu+mloSJJYzTs1V9/cLD5VXXZLNZ5UZLzgQngHVV1P717id3YN2ZXqwHcO63+CuB5wI+qav+A8ZKkMRn2w4+rgLfS+4d7CfAWYCW98yoHPbcyzZX0vpb4ROA+Hj+slgFj6xDqAyVZn2QiycTevXtn0a4kaTZm8yVdJ1fVQ9A74Q58uqrePJuVVdX3p6aTfAT4Ynu6C1jWN3QpsLtND6r/ADgmyRFtb6V//KD1XgVcBbBq1SrvWSZJIzLsnsoLgUf7nj8KLJ/typIc3/f0tfS+phhgC3BekqcmOQFYAdwM3AKsaFd6HUXvZP6WqirgBuB1bfl1wHWz7UeS1K1h91Q+Btyc5PP0DjO9lide1fXPJPkk8ErguCS7gIuBVyY5sb3GTnr3EaOqtie5FriL3tVlF1bVY+11LgKup/elYBurantbxTuBzUneD3wTuHrI9yJJGpFhr/66JMmXgX/XSm+qqm/OsMzrB5QP+A9/VV0CXDKgvpXeDSyn1+/h8SvEJEnzwLCHvwCeDjxYVf8d2NUOU0mS9DPDfqL+YnqHm97VSkcCfzGqpiRJC9OweyqvBV4D/CNAVe1mdpcSS5IOA8OGyqPtiqsCSPKM0bUkSVqohg2Va5P8Ob3PhvwO8BX8wi5J0jTDXv31x+276R8EXgT8YVVtG2lnkqQFZ8ZQSbIIuL6qXg0YJJKkA5rx8Ff7EOLDSZ4zB/1IkhawYT9R/0/AHUm20a4AA6iq3x9JV5KkBWnYUPlSe0iSdEAHDZUkL6yq71bVprlqSJK0cM10TuULUxNJPjviXiRJC9xModL/ZVi/PMpGJEkL30yhUgeYliTpCWY6Uf+yJA/S22M5uk3TnldVPXuk3UmSFpSDhkpVLZqrRiRJC99svk9FkqSDMlQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnTFUJEmdMVQkSZ0xVCRJnRlZqCTZmGRPkjv7as9Nsi3J3e3nsa2eJFckmUxye5KT+5ZZ18bfnWRdX/3lSe5oy1yRJEiSxmrYL+k6FB8FPghc01fbAHy1qi5NsqE9fydwJrCiPV4BXAm8IslzgYuBVfRuaHlrki1VdX8bsx64EdgKrAG+PML3c1havmE8382289Kzx7JeSU/OyPZUqurrwL5p5bXA1Bd+bQLO6atfUz03AsckOR44A9hWVftakGwD1rR5z66qv6+qohdc5yBJGqu5Pqfygqq6D6D9fH6rLwHu7Ru3q9UOVt81oC5JGqP5cqJ+0PmQOoT64BdP1ieZSDKxd+/eQ2xRkjSTuQ6V77dDV7Sfe1p9F7Csb9xSYPcM9aUD6gNV1VVVtaqqVi1evPhJvwlJ0mBzHSpbgKkruNYB1/XVz29Xga0GHmiHx64HTk9ybLtS7HTg+jbvoSSr21Vf5/e9liRpTEZ29VeSTwKvBI5LsoveVVyXAtcmuQD4LnBuG74VOAuYBB4G3gRQVfuSvA+4pY17b1VNnfx/K70rzI6md9WXV35J0piNLFSq6vUHmPWqAWMLuPAAr7MR2DigPgG89Mn0KEnq1nw5US9J+jlgqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6Y6hIkjpjqEiSOmOoSJI6c8S4G1hIlm/40ljWu/PSs8eyXkmarbHsqSTZmeSOJLclmWi15ybZluTu9vPYVk+SK5JMJrk9ycl9r7Oujb87ybpxvBdJ0uPGefjr16vqxKpa1Z5vAL5aVSuAr7bnAGcCK9pjPXAl9EIIuBh4BXAKcPFUEEmSxmM+nVNZC2xq05uAc/rq11TPjcAxSY4HzgC2VdW+qrof2AasmeumJUmPG1eoFPDXSW5Nsr7VXlBV9wG0n89v9SXAvX3L7mq1A9WfIMn6JBNJJvbu3dvh25Ak9RvXifpTq2p3kucD25J86yBjM6BWB6k/sVh1FXAVwKpVqwaOkSQ9eWPZU6mq3e3nHuDz9M6JfL8d1qL93NOG7wKW9S2+FNh9kLokaUzmPFSSPCPJs6amgdOBO4EtwNQVXOuA69r0FuD8dhXYauCBdnjseuD0JMe2E/Snt5okaUzGcfjrBcDnk0yt/xNV9VdJbgGuTXIB8F3g3DZ+K3AWMAk8DLwJoKr2JXkfcEsb996q2jd3b0OSNN2ch0pV3QO8bED9h8CrBtQLuPAAr7UR2Nh1j5KkQzOfLimWJC1whookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpM4aKJKkzhookqTOGiiSpMws+VJKsSfLtJJNJNoy7H0k6nC3oUEmyCPgQcCawEnh9kpXj7UqSDl8LOlSAU4DJqrqnqh4FNgNrx9yTJB22FnqoLAHu7Xu+q9UkSWOQqhp3D4csybnAGVX15vb8DcApVfVfpo1bD6xvT18EfPsQV3kc8INDXHaU7Gt27Gt27Gt2fl77+qWqWjzToCOexArmg13Asr7nS4Hd0wdV1VXAVU92ZUkmqmrVk32drtnX7NjX7NjX7BzufS30w1+3ACuSnJDkKOA8YMuYe5Kkw9aC3lOpqv1JLgKuBxYBG6tq+5jbkqTD1oIOFYCq2gpsnaPVPelDaCNiX7NjX7NjX7NzWPe1oE/US5Lml4V+TkWSNI8YKtMk2ZhkT5I7DzA/Sa5ot4W5PcnJ86SvVyZ5IMlt7fGHc9TXsiQ3JNmRZHuStw0YM+fbbMi+5nybJXlakpuT/EPr648GjHlqkk+17XVTkuXzpK83Jtnbt73ePOq++ta9KMk3k3xxwLw5315D9jWW7ZVkZ5I72jonBswf7e9jVfnoewC/BpwM3HmA+WcBXwYCrAZumid9vRL44hi21/HAyW36WcD/BlaOe5sN2decb7O2DZ7Zpo8EbgJWTxvze8CH2/R5wKfmSV9vBD441/+PtXX/AfCJQf+9xrG9huxrLNsL2Akcd5D5I/19dE9lmqr6OrDvIEPWAtdUz43AMUmOnwd9jUVV3VdV32jTDwE7eOJdDeZ8mw3Z15xr2+DH7emR7TH9xOZaYFOb/gzwqiSZB32NRZKlwNnA/zzAkDnfXkP2NV+N9PfRUJm9+XxrmH/TDl98OclL5nrl7bDDSfT+yu031m12kL5gDNusHTK5DdgDbKuqA26vqtoPPAA8bx70BfDv2yGTzyRZNmD+KPwp8F+Bnx5g/li21xB9wXi2VwF/neTW9O4mMt1Ifx8Nldkb9BfQfPiL7hv0bqPwMuB/AF+Yy5UneSbwWeDtVfXg9NkDFpmTbTZDX2PZZlX1WFWdSO8OEKckeem0IWPZXkP09ZfA8qr6VeArPL53MDJJfhPYU1W3HmzYgNpIt9eQfc359mpOraqT6d29/cIkvzZt/ki3l6Eye0PdGmauVdWDU4cvqvfZnSOTHDcX605yJL1/uD9eVZ8bMGQs22ymvsa5zdo6fwT8DbBm2qyfba8kRwDPYQ4PfR6or6r6YVU90p5+BHj5HLRzKvCaJDvp3YX8tCR/MW3MOLbXjH2NaXtRVbvbzz3A5+ndzb3fSH8fDZXZ2wKc366gWA08UFX3jbupJL8wdRw5ySn0/tv+cA7WG+BqYEdVXXaAYXO+zYbpaxzbLMniJMe06aOBVwPfmjZsC7CuTb8O+Fq1M6zj7GvacffX0DtPNVJV9a6qWlpVy+mdhP9aVf2nacPmfHsN09c4tleSZyR51tQ0cDow/YrRkf4+LvhP1HctySfpXRV0XJJdwMX0TlpSVR+m9+n9s4BJ4GHgTfOkr9cBb02yH/gJcN6of7GaU4E3AHe04/EA7wZe2NfbOLbZMH2NY5sdD2xK7wvmngJcW1VfTPJeYKKqttALw48lmaT3F/d5I+5p2L5+P8lrgP2trzfOQV8DzYPtNUxf49heLwA+3/5WOgL4RFX9VZK3wNz8PvqJeklSZzz8JUnqjKEiSeqMoSJJ6oyhIknqjKEiSeqMoSJJ6oyhIknqjKEiSerM/we1LKLwMk9VDwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c2750cbe0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_train['rating'].plot(kind='hist');\n",
    "print(data_train['rating'].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "pos_data_train = data_train.query(\"rating >= 4\")\n",
    "pos_data_test = data_test.query(\"rating >= 4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because the mean rating is around 3.5, this cut will remove approximately half of the ratings from the datasets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49906"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pos_data_train['rating'].count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5469"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pos_data_test['rating'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Triplet Loss\n",
    "\n",
    "The following section demonstrates how to build a low-rank quadratic interaction model between users and items. The similarity score between a user and an item is defined by the unormalized dot products of their respective embeddings.\n",
    "\n",
    "The matching scores can be use to rank items to recommend to a specific user.\n",
    "\n",
    "Training of the model parameters is achieved by randomly sampling negative items not seen by a pre-selected anchor user. We want the model embedding matrices to be such that the similarity between the user vector and the negative vector is smaller than the similarity between the user vector and the positive item vector. Furthermore we use a margin to further move appart the negative from the anchor user.\n",
    "\n",
    "Here is the architecture of such a triplet architecture. The triplet name comes from the fact that the loss to optimize is defined for triple `(anchor_user, positive_item, negative_item)`:\n",
    "\n",
    "<img src=\"images/rec_archi_implicit_2.svg\" style=\"width: 600px;\" />\n",
    "\n",
    "We call this model a triplet model with bi-linear interactions because the similarity between a user and an item is captured by a dot product of the first level embedding vectors. This is therefore not a deep architecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the actual code that builds the model(s) with shared weights. Note that here we use the cosine similarity instead of unormalized dot products (both seems to yield comparable results)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quality of Ranked Recommendations\n",
    "\n",
    "Now that we have a randomly initialized model we can start computing random recommendations. To assess their quality we do the following for each user:\n",
    "\n",
    "- compute matching scores for items (except the movies that the user has already seen in the training set),\n",
    "- compare to the positive feedback actually collected on the test set using the ROC AUC ranking metric,\n",
    "- average ROC AUC scores across users to get the average performance of the recommender model on the test set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default the model should make predictions that rank the items in random order. The **ROC AUC score** is a ranking score that represents the **expected value of correctly ordering uniformly sampled pairs of recommendations**.\n",
    "\n",
    "A random (untrained) model should yield 0.50 ROC AUC on average. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_size = 64 # embedding size\n",
    "reg_param = 0.01 # regularization parameter lambda\n",
    "learning_rate = 0.01 # learning rate \n",
    "margin = 1.0 # margin \n",
    "\n",
    "# create tensorflow graph\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "    \n",
    "    # setting up random seed\n",
    "    tf.set_random_seed(1234)\n",
    "    \n",
    "    # placeholders\n",
    "    user_input = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    positive_item_input = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    negative_item_input = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    \n",
    "    # variables\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weight = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[max_user_id + 1, embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "\n",
    "        item_weight = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[max_item_id + 1, embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "    # embedding\n",
    "    with tf.name_scope(\"embedding\"):\n",
    "        user_embedding = tf.nn.embedding_lookup(user_weight, user_input)\n",
    "        positive_item_embedding = tf.nn.embedding_lookup(item_weight, positive_item_input)\n",
    "        negative_item_embedding = tf.nn.embedding_lookup(item_weight, negative_item_input)\n",
    "     \n",
    "     # similarity\n",
    "    with tf.name_scope(\"similarity\"):\n",
    "        positive_similarity = tf.reduce_sum(tf.multiply(user_embedding, positive_item_embedding), 1) \n",
    "        negative_similarity = tf.reduce_sum(tf.multiply(user_embedding, negative_item_embedding), 1) \n",
    "        \n",
    "    # loss \n",
    "    with tf.name_scope(\"loss\"):\n",
    "        triplet_loss = tf.maximum(negative_similarity - positive_similarity + margin, 0)\n",
    "        loss = tf.reduce_mean(triplet_loss)\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sample_triplets(pos_data, max_item_id, random_seed=0):\n",
    "    \"\"\"Sample negatives at random\"\"\"\n",
    "    rng = np.random.RandomState(random_seed)\n",
    "    user_ids = pos_data['user_id'].values\n",
    "    pos_item_ids = pos_data['item_id'].values\n",
    "\n",
    "    neg_item_ids = rng.randint(low=1, high=max_item_id + 1,\n",
    "                               size=len(user_ids))\n",
    "    return [ user_ids, pos_item_ids,neg_item_ids]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the Triplet Model\n",
    "\n",
    "Let's now fit the parameters of the model by sampling triplets: for each user, select a movie in the positive feedback set of that user and randomly sample another movie to serve as negative item.\n",
    "\n",
    "Note that this sampling scheme could be improved by removing items that are marked as positive in the data to remove some label noise. In practice this does not seem to be a problem though."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train the triplet model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration : 0 train loss: 0.992 , valid loss 1.000 , ROC auc 0.5014\n",
      "iteration : 10 train loss: 0.670 , valid loss 0.751 , ROC auc 0.8609\n",
      "iteration : 20 train loss: 0.325 , valid loss 0.361 , ROC auc 0.8819\n",
      "iteration : 30 train loss: 0.273 , valid loss 0.283 , ROC auc 0.9002\n",
      "iteration : 40 train loss: 0.223 , valid loss 0.237 , ROC auc 0.9179\n",
      "iteration : 50 train loss: 0.194 , valid loss 0.226 , ROC auc 0.9238\n",
      "iteration : 60 train loss: 0.172 , valid loss 0.233 , ROC auc 0.9257\n",
      "iteration : 70 train loss: 0.159 , valid loss 0.219 , ROC auc 0.9264\n",
      "iteration : 80 train loss: 0.146 , valid loss 0.236 , ROC auc 0.9257\n",
      "iteration : 90 train loss: 0.135 , valid loss 0.236 , ROC auc 0.9246\n",
      "iteration : 100 train loss: 0.126 , valid loss 0.248 , ROC auc 0.9236\n",
      "iteration : 110 train loss: 0.120 , valid loss 0.242 , ROC auc 0.9224\n",
      "iteration : 120 train loss: 0.116 , valid loss 0.268 , ROC auc 0.9211\n",
      "iteration : 130 train loss: 0.110 , valid loss 0.242 , ROC auc 0.9197\n",
      "iteration : 140 train loss: 0.108 , valid loss 0.253 , ROC auc 0.9184\n",
      "overfit !\n"
     ]
    }
   ],
   "source": [
    "n_epochs = 1000\n",
    "losses_train = []\n",
    "losses_val = []\n",
    "    \n",
    "with tf.Session(graph=g) as sess:\n",
    "    # initializer\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    def check_overfit(validation_loss):\n",
    "        n = len(validation_loss)\n",
    "        if n < 5:\n",
    "            return False\n",
    "        count = 0 \n",
    "        for i in range(n-4, n):\n",
    "            if validation_loss[i] < validation_loss[i-1]:\n",
    "                count += 1\n",
    "            if count >=2:\n",
    "                return False\n",
    "        return True\n",
    "            \n",
    "    for i in range(n_epochs):\n",
    "        triplet_inputs_train = sample_triplets(pos_data_train, max_item_id,random_seed=i)\n",
    "        triplet_inputs_val = sample_triplets(pos_data_test, max_item_id,random_seed=i+1)\n",
    "        \n",
    "        train_input_dict = {user_input: triplet_inputs_train[0]\n",
    "                            , positive_item_input: triplet_inputs_train[1]\n",
    "                            , negative_item_input: triplet_inputs_train[2]}\n",
    "        \n",
    "        val_input_dict = {user_input: triplet_inputs_val[0]\n",
    "                            , positive_item_input: triplet_inputs_val[1]\n",
    "                            , negative_item_input: triplet_inputs_val[2]}\n",
    "        sess.run([train_ops], feed_dict=train_input_dict)\n",
    "        \n",
    "        if i % 10 == 0:\n",
    "            loss_train = sess.run(loss, feed_dict=train_input_dict)\n",
    "            loss_val = sess.run(loss, feed_dict=val_input_dict)\n",
    "\n",
    "            losses_train.append(loss_train)\n",
    "            losses_val.append(loss_val)\n",
    "\n",
    "            # check early stopping \n",
    "            if(check_overfit(losses_val)):\n",
    "                print('overfit !')\n",
    "                break\n",
    "\n",
    "             \n",
    "            # calculate AUC Score \n",
    "            \"\"\"Compute the ROC AUC for each user and average over users\"\"\"\n",
    "            max_user_id = max(pos_data_train['user_id'].max(), pos_data_test['user_id'].max())\n",
    "            max_item_id = max(pos_data_train['item_id'].max(), pos_data_test['item_id'].max())\n",
    "            user_auc_scores = []\n",
    "            for user_id in range(1, max_user_id + 1):\n",
    "                pos_item_train = pos_data_train[pos_data_train['user_id'] == user_id]\n",
    "                pos_item_test = pos_data_test[pos_data_test['user_id'] == user_id]\n",
    "\n",
    "                # Consider all the items already seen in the training set\n",
    "                all_item_ids = np.arange(1, max_item_id + 1)\n",
    "                items_to_rank = np.setdiff1d(all_item_ids, pos_item_train['item_id'].values)\n",
    "\n",
    "                # Ground truth: return 1 for each item positively present in the test set\n",
    "                # and 0 otherwise.\n",
    "                expected = np.in1d(items_to_rank, pos_item_test['item_id'].values)\n",
    "\n",
    "                if np.sum(expected) >= 1:\n",
    "                    # At least one positive test value to rank\n",
    "                    repeated_user_id = np.empty_like(items_to_rank)\n",
    "                    repeated_user_id.fill(user_id)\n",
    "                    predicted = sess.run(positive_similarity, feed_dict={user_input : repeated_user_id, \n",
    "                                                            positive_item_input : items_to_rank})\n",
    "                    user_auc_scores.append(roc_auc_score(expected, predicted))\n",
    "\n",
    "            print(\"iteration : %d train loss: %.3f , valid loss %.3f , ROC auc %.4f\" % (i,loss_train, loss_val,np.mean(user_auc_scores)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a Deep Matching Model on Implicit Feedback\n",
    "\n",
    "\n",
    "Instead of using hard-coded cosine similarities to predict the match of a `(user_id, item_id)` pair, we can instead specify a deep neural network based parametrisation of the similarity. The parameters of that matching model are also trained with the margin comparator loss:\n",
    "\n",
    "<img src=\"images/rec_archi_implicit_1.svg\" style=\"width: 600px;\" />\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"embedding/user_embedding:0\", shape=(?, 32), dtype=float32)\n",
      "Tensor(\"embedding/positive_item_embedding:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"embedding/negative_item_embedding:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"concatenation/concat:0\", shape=(?, 96), dtype=float32)\n",
      "Tensor(\"concatenation/concat_1:0\", shape=(?, 96), dtype=float32)\n",
      "Tensor(\"fc_1/hidden_output_positive:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"fc_1/hidden_output_negative:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"dropout/hidden_output_drop_positive/mul:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"dropout/hidden_output_drop_negative/mul:0\", shape=(?, 64), dtype=float32)\n",
      "Tensor(\"fc_2/positive_prediction:0\", shape=(?, 1), dtype=float32)\n",
      "Tensor(\"fc_2/negative_prediction:0\", shape=(?, 1), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "user_embedding_size = 32\n",
    "item_embedding_size = 64\n",
    "num_hidden = 64\n",
    "\n",
    "reg_param = 0.01\n",
    "learning_rate = 0.01\n",
    "n_users = max_user_id + 1\n",
    "n_items = max_item_id + 1\n",
    "\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "    \n",
    "    # setting up random seed\n",
    "    tf.set_random_seed(1234)\n",
    "\n",
    "    user_input = tf.placeholder(shape=[None,1], dtype=tf.int64, name='user_input')\n",
    "    positive_item_input = tf.placeholder(shape=[None,1], dtype=tf.int64, name='positive_item_input')\n",
    "    negative_item_input = tf.placeholder(shape=[None,1], dtype=tf.int64, name='negative_item_input')\n",
    "    \n",
    "    l2_loss = tf.constant(0.0)\n",
    "    \n",
    "    # embeddding layer\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weights = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[n_users, user_embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        item_weights = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[n_items, item_embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        user_embedding = tf.squeeze(tf.nn.embedding_lookup(user_weights, user_input),axis=1, name='user_embedding')\n",
    "        positive_item_embedding = tf.squeeze(tf.nn.embedding_lookup(item_weights, positive_item_input),axis=1, name='positive_item_embedding')\n",
    "        negative_item_embedding = tf.squeeze(tf.nn.embedding_lookup(item_weights, negative_item_input),axis=1, name='negative_item_embedding')\n",
    "        \n",
    "        l2_loss += tf.nn.l2_loss(user_weights)\n",
    "        l2_loss += tf.nn.l2_loss(item_weights)\n",
    "        \n",
    "        \n",
    "        print(user_embedding)\n",
    "        print(positive_item_embedding)\n",
    "        print(negative_item_embedding)\n",
    "        \n",
    "    \n",
    "    # combine inputs\n",
    "    with tf.name_scope('concatenation'):\n",
    "        positive_embeddings_pair = tf.concat([user_embedding, positive_item_embedding], axis=1)\n",
    "        negative_embeddings_pair = tf.concat([user_embedding, negative_item_embedding], axis=1)\n",
    "        print(positive_embeddings_pair)\n",
    "        print(negative_embeddings_pair)\n",
    "        \n",
    "    # fc-1\n",
    "    \n",
    "    with tf.name_scope(\"fc_1\"):\n",
    "        W_fc_1 = tf.get_variable(\n",
    "            \"W_hidden\",\n",
    "            shape=[user_embedding_size + item_embedding_size, num_hidden],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_1 = tf.Variable(tf.constant(0.1, shape=[num_hidden]), name=\"b\")\n",
    "        hidden_output_positive = tf.nn.relu(tf.nn.xw_plus_b(positive_embeddings_pair, W_fc_1, b_fc_1), name='hidden_output_positive')\n",
    "        hidden_output_negative = tf.nn.relu(tf.nn.xw_plus_b(negative_embeddings_pair, W_fc_1, b_fc_1), name='hidden_output_negative')\n",
    "        \n",
    "        l2_loss += tf.nn.l2_loss(W_fc_1)\n",
    "        print(hidden_output_positive)\n",
    "        print(hidden_output_negative)\n",
    "        \n",
    "    # dropout\n",
    "    with tf.name_scope(\"dropout\"):\n",
    "        h_drop_positive = tf.nn.dropout(hidden_output_positive, 0.8, name=\"hidden_output_drop_positive\")\n",
    "        h_drop_negative = tf.nn.dropout(hidden_output_negative, 0.8, name=\"hidden_output_drop_negative\")\n",
    "        print(h_drop_positive)\n",
    "        print(h_drop_negative)\n",
    "    \n",
    "    # fc-2\n",
    "    with tf.name_scope(\"fc_2\"):\n",
    "        W_fc_2 = tf.get_variable(\n",
    "            \"W_output\",\n",
    "            shape=[num_hidden,1],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_2 = tf.Variable(tf.constant(0.1, shape=[1]), name=\"b\")\n",
    "        positive_prediction = tf.nn.xw_plus_b(h_drop_positive, W_fc_2, b_fc_2, name='positive_prediction')\n",
    "        negative_prediction = tf.nn.xw_plus_b(h_drop_negative, W_fc_2, b_fc_2, name='negative_prediction')\n",
    "        \n",
    "        l2_loss += tf.nn.l2_loss(W_fc_2)\n",
    "        print(positive_prediction)\n",
    "        print(negative_prediction)\n",
    "\n",
    "    # loss\n",
    "    with tf.name_scope(\"loss\"):\n",
    "        triplet_loss = tf.maximum(negative_prediction - positive_prediction + margin, 0)\n",
    "        loss = tf.reduce_mean(triplet_loss) + reg_param * l2_loss\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration : 0 train loss: 1.882 , valid loss 1.881 , ROC auc 0.6317\n",
      "iteration : 10 train loss: 0.787 , valid loss 0.773 , ROC auc 0.8665\n",
      "iteration : 20 train loss: 0.591 , valid loss 0.593 , ROC auc 0.8591\n",
      "iteration : 30 train loss: 0.548 , valid loss 0.531 , ROC auc 0.8638\n",
      "iteration : 40 train loss: 0.528 , valid loss 0.511 , ROC auc 0.8635\n",
      "iteration : 50 train loss: 0.521 , valid loss 0.512 , ROC auc 0.8635\n",
      "overfit !\n"
     ]
    }
   ],
   "source": [
    "n_epochs = 1000\n",
    "losses_train = []\n",
    "losses_val = []\n",
    "\n",
    "with tf.Session(graph=g) as sess:\n",
    "    # initializer\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    def check_overfit(validation_loss):\n",
    "        n = len(validation_loss)\n",
    "        if n < 5:\n",
    "            return False\n",
    "        count = 0 \n",
    "        for i in range(n-4, n):\n",
    "            if validation_loss[i] < validation_loss[i-1]:\n",
    "                count += 1\n",
    "            if count >=3:\n",
    "                return False\n",
    "        return True\n",
    "            \n",
    "    for i in range(n_epochs):\n",
    "        triplet_inputs_train = sample_triplets(pos_data_train, max_item_id,random_seed=i)\n",
    "        triplet_inputs_val = sample_triplets(pos_data_test, max_item_id,random_seed=i+1)\n",
    "        \n",
    "        train_input_dict = {user_input: triplet_inputs_train[0].reshape([-1,1])\n",
    "                            , positive_item_input: triplet_inputs_train[1].reshape([-1,1])\n",
    "                            , negative_item_input: triplet_inputs_train[2].reshape([-1,1])}\n",
    "        \n",
    "        val_input_dict = {user_input: triplet_inputs_val[0].reshape([-1,1])\n",
    "                            , positive_item_input: triplet_inputs_val[1].reshape([-1,1])\n",
    "                            , negative_item_input: triplet_inputs_val[2].reshape([-1,1])}\n",
    "        sess.run([train_ops], feed_dict=train_input_dict)\n",
    "        \n",
    "        if i % 10 == 0:\n",
    "            loss_train = sess.run(loss, feed_dict=train_input_dict)\n",
    "            loss_val = sess.run(loss, feed_dict=val_input_dict)\n",
    "\n",
    "            losses_train.append(loss_train)\n",
    "            losses_val.append(loss_val)\n",
    "\n",
    "            # check early stopping \n",
    "            if(check_overfit(losses_val)):\n",
    "                print('overfit !')\n",
    "                break\n",
    "\n",
    "            \n",
    "            # calculate AUC Score \n",
    "            \"\"\"Compute the ROC AUC for each user and average over users\"\"\"\n",
    "            max_user_id = max(pos_data_train['user_id'].max(), pos_data_test['user_id'].max())\n",
    "            max_item_id = max(pos_data_train['item_id'].max(), pos_data_test['item_id'].max())\n",
    "            user_auc_scores = []\n",
    "            for user_id in range(1, max_user_id + 1):\n",
    "                pos_item_train = pos_data_train[pos_data_train['user_id'] == user_id]\n",
    "                pos_item_test = pos_data_test[pos_data_test['user_id'] == user_id]\n",
    "\n",
    "                # Consider all the items already seen in the training set\n",
    "                all_item_ids = np.arange(1, max_item_id + 1)\n",
    "                items_to_rank = np.setdiff1d(all_item_ids, pos_item_train['item_id'].values)\n",
    "\n",
    "                # Ground truth: return 1 for each item positively present in the test set\n",
    "                # and 0 otherwise.\n",
    "                expected = np.in1d(items_to_rank, pos_item_test['item_id'].values)\n",
    "\n",
    "                if np.sum(expected) >= 1:\n",
    "                    # At least one positive test value to rank\n",
    "                    repeated_user_id = np.empty_like(items_to_rank)\n",
    "                    repeated_user_id.fill(user_id)\n",
    "                    predicted = sess.run(positive_prediction, feed_dict={user_input : repeated_user_id.reshape([-1,1]), \n",
    "                                                            positive_item_input : items_to_rank.reshape([-1,1])})\n",
    "                    user_auc_scores.append(roc_auc_score(expected, predicted))\n",
    "\n",
    "            print(\"iteration : %d train loss: %.3f , valid loss %.3f , ROC auc %.4f\" % (i,loss_train, loss_val,np.mean(user_auc_scores)))\n",
    "\n",
    "\n",
    "            \n",
    "            \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Possible Extensions\n",
    "\n",
    "You can implement any of the following ideas if you want to get a deeper understanding of recommender systems.\n",
    "\n",
    "\n",
    "### Leverage User and Item metadata\n",
    "\n",
    "As we did for the Explicit Feedback model, it's also possible to extend our models to take additional user and item metadata as side information when computing the match score.\n",
    "\n",
    "\n",
    "### Better Ranking Metrics\n",
    "\n",
    "In this notebook we evaluated the quality of the ranked recommendations using the ROC AUC metric. This score reflect the ability of the model to correctly rank any pair of items (sampled uniformly at random among all possible items).\n",
    "\n",
    "In practice recommender systems will only display a few recommendations to the user (typically 1 to 10). It is typically more informative to use an evaluatio metric that characterize the quality of the top ranked items and attribute less or no importance to items that are not good recommendations for a specific users. Popular ranking metrics therefore include the **Precision at k** and the **Mean Average Precision**.\n",
    "\n",
    "\n",
    "\n",
    "### Hard Negatives Sampling\n",
    "\n",
    "In this experiment we sampled negative items uniformly at random. However, after training the model for a while, it is possible that the vast majority of sampled negatives have a similarity already much lower than the positive pair and that the margin comparator loss sets the majority of the gradients to zero effectively wasting a lot of computation.\n",
    "\n",
    "Given the current state of the recsys model we could sample harder negatives with a larger likelihood to train the model better closer to its decision boundary. This strategy is implemented in the WARP loss [1].\n",
    "\n",
    "The main drawback of hard negative sampling is increasing the risk of sever overfitting if a significant fraction of the labels are noisy.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}