lab_day1/day1_code00_basic_python_[EXERCISE].ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jaH0umpACXMW"
      },
      "source": [
        "## Trying out Jupyter Notebooks with basic relevant Python\n",
        "\n",
        "Let's get acquainted with Python (3) and interactive Jupyter Notebooks (ipynb), our main working tools throughout the course.\n",
        "\n",
        "First of all, in an `ipynb` file we distinguish:\n",
        "\n",
        "- text chunks: where we write instructions, comments, explanations (like this one!)\n",
        "- code chunks: where we write Python instructions to be executed\n",
        "\n",
        "As for `Python`, we will mainly use three fundamental libraries:\n",
        "\n",
        "1. `numpy`: to work with arrays (vectors, matrices)\n",
        "2. `pandas`: to work with dataframes (reading and manipulating datasets)\n",
        "3. `matplotlib`: to visualize data and results (making plots)\n",
        "\n",
        "In addition, we'll use `Keras` for deep learning: we'll see more on this later.\n",
        "\n",
        "This is not a course on Python, not even an introduction: just a few notes to get some basic acquaintance with these libraries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aiwyg8qDfr8Z"
      },
      "source": [
        "# How to proceed?\n",
        "Tell us your preference!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g_Zh_UVRMGAO"
      },
      "source": [
        "### Numpy\n",
        "\n",
        "We start by importing the library and creating a vector (one-dimensional array)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kr0ZWKkUCUHa"
      },
      "source": [
        "## import libraries\n",
        "import numpy as np\n",
        "\n",
        "arr_1d = np.array([1, 2, 3, 4, 5, 0, 7, 1, 9])\n",
        "print(arr_1d)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1g5lIYQ7MwQQ"
      },
      "source": [
        "We can use the array attribute `shape` to get the dimensions of the array"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qdJmusLYMiVa"
      },
      "source": [
        "arr_1d.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T_FZlsUDM3lN"
      },
      "source": [
        "We see that by default `numpy` does not specify the second dimension for one-dimensional arrays (vectors). If you want to make this explicit (which may turn out to be helpful for some array operations and to avoid ambiguous results), you can use the **function** `reshape()`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RHIOgFTEMoU7"
      },
      "source": [
        "arr_1d = arr_1d.reshape(len(arr_1d),1)\n",
        "print(arr_1d.shape)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5xxg35CINw6j"
      },
      "source": [
        "We can also build arrays with more dimensions (matrices, \"tensors\"): in deep learning, mastering the dimensions of tensors (multidimensional arrays) is very important!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "49Los7EiMhCk"
      },
      "source": [
        "arr_2d = np.array([[1, 2, 3], [4, 5, 6]])\n",
        "print(arr_2d)\n",
        "print(arr_2d.shape) ## 2 rows, 3 columns"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YuiabkJbNsfi"
      },
      "source": [
        "arr_3d = np.array([[[1, 2, 3], [4, 5, 6]], [[1, 2, 3], [4, 5, 6]]])\n",
        "print(arr_3d)\n",
        "print(arr_3d.shape) ## 2 layers, 2 row, 3 columns"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NkAkRRmpQkF_"
      },
      "source": [
        "value = arr_3d[0,1,2] ## get first layer, second row, third element (indexing in Python starts at 0!)\n",
        "print(value)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L8PSvRstRI_x"
      },
      "source": [
        "- use array indices for slicing: `[start:end]`\n",
        "- a step argument can be added: `[start:end:step]`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zot89l11RHIS"
      },
      "source": [
        "print(arr_1d[:]) ## entire array\n",
        "print(arr_1d[3:6]) ## elements 4, 5 and 6 of the array\n",
        "print(arr_1d[:3]) ## first three elements of the array --> equivalent to arr_1d[0:3]\n",
        "print(arr_1d[7:]) ## last two elements of the array --> equivalent to arr_1d[7:len(arr_1d)]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "53bjX-f9Vv0P"
      },
      "source": [
        "More on `numpy` can be found for instance <a href='https://numpy.org/devdocs/user/quickstart.html'>here</a>\n",
        "\n",
        "### Pandas\n",
        "\n",
        "Python library to work with tabular data (multidimensional), in the form of mainly **dataframes**.\n",
        "\n",
        "We load the library and create a first dataframe:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-ZcspfLkV3wM"
      },
      "source": [
        "import pandas as pd\n",
        "\n",
        "df = pd.DataFrame(np.random.randn(8, 4), columns=list('ABCD')) ## list() constructor converts string to a list of characters\n",
        "df"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4MaVObw-ZaBA"
      },
      "source": [
        "With the dataframe attribute `dtypes` we can get the data type of each column in a Pandas dataframe:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3wpHa2F9WJxJ"
      },
      "source": [
        "df.dtypes"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x0UPVJWQaGU6"
      },
      "source": [
        "The Pandas function `head()` will show the first few rows of the dataframe; the attribute `columns` returns the column names."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5Ztb9mweZt1S"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DYWIzjVNaVDJ"
      },
      "source": [
        "df.columns"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3N_MY6Eah7w"
      },
      "source": [
        "With the function `to_numpy()` we convert a dataframe to an array; then we can slice it as we already saw for numpy arrays"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_DKV0Mh1aWmF"
      },
      "source": [
        "arr = df.to_numpy()\n",
        "arr"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y1BEFYf8adzB"
      },
      "source": [
        "arr[:,1] ## get second column"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3lRZp5vPa__X"
      },
      "source": [
        "You can also slice directly the Pandas dataframe:\n",
        "\n",
        "1. by column name"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Xy2ny28NbGY1"
      },
      "source": [
        "df['A']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DPtfUDRpeTf2"
      },
      "source": [
        "2. by slicing by rows"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EEk2sk9BbJia"
      },
      "source": [
        "df[0:2]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uZB63KcieDBX"
      },
      "source": [
        "3. by row and/or column names using the `.loc` syntax"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5eRtTHnrbUxp"
      },
      "source": [
        "print(df.loc[df.index.values[0]])\n",
        "df.loc[:, ['B','C']]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uU7cZCApeioJ"
      },
      "source": [
        "4. by position using the `.iloc` syntax"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "obQdZOOGeZiK"
      },
      "source": [
        "df.iloc[0:2,2:4]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MH47bz95exf8"
      },
      "source": [
        "More on `pandas` <a href='https://pandas.pydata.org/pandas-docs/stable/getting_started/tutorials.html'>here</a>\n",
        "\n",
        "### Matplotlib\n",
        "\n",
        "`matplotlib` is a versatile Python library for data visualization which allows you to produce a very large variety of high-quality plots.\n",
        "We import the module `pyplot` from the library and produce a first basic plot:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BMJTACwbezSB"
      },
      "source": [
        "from matplotlib import pyplot as plt\n",
        "\n",
        "x = np.array([1, 2, 3, 4])\n",
        "y = np.array([1, 4, 9, 16])\n",
        "plt.xlabel('x-axis')\n",
        "plt.ylabel('y-axis')\n",
        "plt.plot(x,y)\n",
        "\n",
        "ax = plt.gca()\n",
        "ax.set_xlim([0, 110])\n",
        "ax.set_ylim([-20, 20])\n",
        "\n",
        "plt.show() ## not strictly necessary in interactive mode (as ipython or jupyter notebooks)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OuzNetZFm9ev"
      },
      "source": [
        "Instead of a diagram (line plot), we can also make a **scatter plot**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "inJoWtz3lK-2"
      },
      "source": [
        "x = np.array([1, 2, 3, 4, 0, 2, 7, 4, 10])\n",
        "y = np.array([1, 4, 9, 16, 1, 3, 9, 8, 15])\n",
        "plt.scatter(x,y)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aZHXrgO-nK2c"
      },
      "source": [
        "For categorical data, a **barplot** can be used:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KTJzqS5Pk2iI"
      },
      "source": [
        "names = ['A', 'B', 'c']\n",
        "values = [1, 10, 100]\n",
        "\n",
        "plt.figure(figsize=(9, 3))\n",
        "\n",
        "plt.plot(131)\n",
        "plt.bar(names, values)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wTdluJEYnlVc"
      },
      "source": [
        "We can plot a distribution from synthetic data using a **histogram**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6E8VIceinbwM"
      },
      "source": [
        "mu, sigma = 100, 15\n",
        "x = mu + sigma * np.random.randn(10000)\n",
        "\n",
        "# the histogram of the data\n",
        "plt.hist(x, 50, density=1, facecolor='g', alpha=0.75)\n",
        "\n",
        "plt.xlabel('Smarts')\n",
        "plt.ylabel('Probability')\n",
        "plt.title('Histogram of IQ')\n",
        "plt.text(60, .025, r'$\\mu=100,\\ \\sigma=15$')\n",
        "plt.axis([40, 160, 0, 0.03])\n",
        "plt.grid(True)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RJquOQRSxRRa"
      },
      "source": [
        "You can also plot directly from a Pandas dataframe:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NimJTo8RtGw2"
      },
      "source": [
        "df = pd.DataFrame(np.random.randn(1000, 4), columns=list('ABCD'))\n",
        "df"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tP-ZI1qctYAC"
      },
      "source": [
        "plt.plot(df)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6DhtEOBstN0k"
      },
      "source": [
        "df = df.cumsum() ## cumulative sums per columns\n",
        "df"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kAmwuD0xthZa"
      },
      "source": [
        "plt.figure()\n",
        "df.plot()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8-5hbDI_xpMz"
      },
      "source": [
        "In Pandas, plot functions can be called directly on dataframes, e.g. using the function `plot.bar()` (we saw an example earlier with `df.plot()`)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XQXb5HE9xqBg"
      },
      "source": [
        "df = pd.DataFrame(np.random.rand(10, 4), columns=['a', 'b', 'c', 'd'])\n",
        "df.plot.bar();"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RdxtdEkdyDVS"
      },
      "source": [
        "More on `matplotlib` <a href='https://matplotlib.org/tutorials/introductory/pyplot.html'>here</a>\n",
        "\n",
        "## Exercise 0.1: do-it-yourself!\n",
        "\n",
        "Now it is your turn to put together what you just learnt:\n",
        "\n",
        "1. get your data\n",
        "  - create a numpy array (e.g. `np.random.randn()`); or\n",
        "  - create a pandas dataframe (e.g. `pd.DataFrame()`); or\n",
        "  - import a built-in dataset\n",
        "    - e.g. `from sklearn import datasets`\n",
        "    - `datasets.load_wine().data` (the data)\n",
        "    - `datasets.load_wine().feature_names` (the column names)\n",
        "    - (more on `sklearn` [here](https://scikit-learn.org/stable/))\n",
        "  - or a combination of these\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#################\n",
        "## your code here\n",
        "#################\n",
        "\n",
        "## 1. sample, generate or import your data\n"
      ],
      "metadata": {
        "id": "seBHBlS08hmy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "## 2. convert to a pandas dataframe (e.g. pd.DataFrame())"
      ],
      "metadata": {
        "id": "0-K-le4E3o-8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "## 3. give names to dataframe columns (e.g. df.columns = list_of_names)"
      ],
      "metadata": {
        "id": "ugliN6HO3qc_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "## 4. subset the dataframe keeping only the columns that you wish to use"
      ],
      "metadata": {
        "id": "u7-3fnLT3sGv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "## 5. make sure that the columns you selected are of the correct data type for plotting"
      ],
      "metadata": {
        "id": "-sIDVkC43tU_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "2. plot the data"
      ],
      "metadata": {
        "id": "gOOMxcog7oXA"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7pgaMu_wyUtw"
      },
      "source": [
        "#################\n",
        "## your code here\n",
        "#################"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}