keyworth_materials/Numpy_intro_keyworth_feb2019.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy - a basic introduction\n",
    "\n",
    "## What is it\n",
    "\n",
    "See: https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.ndarray.html\n",
    "\n",
    "An ndarray is a (usually fixed-size) multidimensional container of items of the same type and size. The number of dimensions and items in an array is defined by its shape, which is a tuple of N positive integers that specify the sizes of each dimension. The type of items in the array is specified by a separate data-type object (dtype), one of which is associated with each ndarray.\n",
    "\n",
    "As with other container objects in Python, the contents of an ndarray can be accessed and modified by indexing or slicing the array (using, for example, N integers), and via the methods and attributes of the ndarray.\n",
    "\n",
    "Different ndarrays can share the same data, so that changes made in one ndarray may be visible in another. That is, an ndarray can be a “view” to another ndarray, and the data it is referring to is taken care of by the “base” ndarray. ndarrays can also be views to memory owned by Python strings or objects implementing the buffer or array interfaces.\n",
    "\n",
    "## Why would you use it?\n",
    "\n",
    "* It's fast!\n",
    "* Much more memory efficient than using basic Python lists\n",
    "* Built for matrix operations/manipulations - moving windows/convolutions, raster work, image analysis etc.\n",
    "* Creates the foundation for many other packages e.g. pandas\n",
    "\n",
    "## Aim of this tutorial\n",
    "\n",
    "* Basic introduction\n",
    "* Demonstrate array creation\n",
    "* Visulaise arrays\n",
    "* Add/Multiply/Divide\n",
    "* Plan for taking things further\n",
    "\n",
    "## Start with some imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.axes_grid1 import make_axes_locatable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define some plotting functions (to help with the examples - don't worry about these for now)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_single_plot(arr, title):\n",
    "\t# see https://stackoverflow.com/questions/23876588/matplotlib-colorbar-in-each-subplot\n",
    "\tfig=plt.figure()\n",
    "\tax1=fig.add_subplot(111)\n",
    "\tim1=ax1.imshow(arr)\n",
    "\n",
    "\tax1.set_title(title) \n",
    "\n",
    "\tdivider = make_axes_locatable(ax1)\n",
    "\tcax = divider.append_axes('right', size='5%', pad=0.05)\n",
    "\tfig.colorbar(im1, cax=cax, orientation='vertical')\n",
    "\n",
    "\tplt.show()\n",
    "\n",
    "def create_plot(arr_vals, arr_vals_5, arr_vals_x10, arr_vals_div3):\n",
    "\t# see https://stackoverflow.com/questions/23876588/matplotlib-colorbar-in-each-subplot\n",
    "\tfig=plt.figure()\n",
    "\tax1=fig.add_subplot(221)\n",
    "\tax2=fig.add_subplot(222)\n",
    "\tax3=fig.add_subplot(223)\n",
    "\tax4=fig.add_subplot(224)\n",
    "\tim1=ax1.imshow(arr_vals)\n",
    "\tim2=ax2.imshow(arr_vals_5)\n",
    "\tim3=ax3.imshow(arr_vals_x10)\n",
    "\tim4=ax4.imshow(arr_vals_div3)\n",
    "\n",
    "\tax1.set_title(\"Before addition\") \n",
    "\tax2.set_title(\"After addition\")\n",
    "\tax3.set_title(\"After multiplication\")\n",
    "\tax4.set_title(\"After division\")\n",
    "\n",
    "\tdivider = make_axes_locatable(ax1)\n",
    "\tcax = divider.append_axes('right', size='5%', pad=0.05)\n",
    "\tfig.colorbar(im1, cax=cax, orientation='vertical')\n",
    "\n",
    "\tdivider = make_axes_locatable(ax2)\n",
    "\tcax = divider.append_axes('right', size='5%', pad=0.05)\n",
    "\tfig.colorbar(im2, cax=cax, orientation='vertical')\n",
    "\n",
    "\tdivider = make_axes_locatable(ax3)\n",
    "\tcax = divider.append_axes('right', size='5%', pad=0.05)\n",
    "\tfig.colorbar(im3, cax=cax, orientation='vertical')\n",
    "\n",
    "\tdivider = make_axes_locatable(ax4)\n",
    "\tcax = divider.append_axes('right', size='5%', pad=0.05)\n",
    "\tfig.colorbar(im4, cax=cax, orientation='vertical')\n",
    "\n",
    "\tplt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a 1D ndarray object / an array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.arange(10) \n",
    "print(a)\n",
    "print(type(a))\n",
    "print(a.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a 2D array\n",
    "\n",
    "We'll now create some 2-dimesnional ndarrays, with one full of ones, one with custom values and one with random values. Each array we will create will have 3 columns and 2 rows.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_ones=np.ones([3,3])\n",
    "arr_vals=np.array([[1,2,3],[4,5,6],[7,8,9]])\n",
    "arr_random=np.random.rand(3,3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now print the arrays out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"arr_ones:\")\n",
    "print(arr_ones)\n",
    "print(\"arr_vals:\")\n",
    "print(arr_vals)\n",
    "print(\"arr_random:\")\n",
    "print(arr_random)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's look at the type and dimensions of one of the arrays (`arr_ones`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"arr_ones type:\")\n",
    "print(type(arr_ones))\n",
    "print(\"arr_ones shape:\")\n",
    "print(arr_ones.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's see these arrays as images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_single_plot(arr_ones, \"Ones array\")\n",
    "create_single_plot(arr_vals, \"Custom array\")\n",
    "create_single_plot(arr_random, \"Random value array\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding, multiplying and dividing\n",
    "\n",
    "To add a value to every element of an array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_vals_5=arr_vals+5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To multiply every element of an array by a value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_vals_x10=arr_vals*10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To divide every element of an array by a value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arr_vals_div3=arr_vals/3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the results (note the colorbar):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_plot(arr_vals, arr_vals_5, arr_vals_x10, arr_vals_div3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indexing and slicing\n",
    "\n",
    "To access specific values, rows or columns, we can use indexing and slicing - remember that the first element is 0 (not 1). Loads of info is available here: https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html\n",
    "\n",
    "For a quick demo using our 1d array, let's get a specific index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(a)\n",
    "print(a[5]) # specific index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking this further you can also take a slice between index psotions with a set step interval:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(a[2:7:2]) # (start:stop:step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a quick demo using our 2d array, let's get the first row and first column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Original array (custom):\")\n",
    "print(arr_vals)\n",
    "\n",
    "print(\"1st row:\")\n",
    "print(arr_vals[0,:])\n",
    "print(\"1st column:\")\n",
    "print(arr_vals[:,0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also access multiple columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"1st column:\")\n",
    "print(arr_vals[:,0]) \n",
    "\n",
    "print(\"2nd column:\")\n",
    "print(arr_vals[:,1]) \n",
    "\n",
    "print(\"Last 2 columns:\")\n",
    "print(arr_vals[:,1:3]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"1st row:\")\n",
    "print(arr_vals[0,...]) \n",
    "\n",
    "print(\"1st 2 rows:\")\n",
    "print(arr_vals[0:2,...]) \n",
    "\n",
    "print(\"Last 2 rows:\")\n",
    "print(arr_vals[1:3,...])\n",
    "\n",
    "print(\"Just the middle row\")\n",
    "print(arr_vals[1:2,...])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Flip\n",
    "\n",
    "Want to flip an array? You can do that too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Original array:\")\n",
    "print(a)\n",
    "\n",
    "print(\"Flipped array:\")\n",
    "print(a[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other stuff:\n",
    "\n",
    "* Use [stackoverflow](https://stackoverflow.com) if you get stuck - someone has probably had the same problem before!\n",
    "* Numpy has loads of in-built functions..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Square root:\")\n",
    "print(np.sqrt(arr_vals))\n",
    "print(np.round(np.sqrt(arr_vals)))\n",
    "\n",
    "print(\"Median:\")\n",
    "print(np.median(arr_vals))\n",
    "\n",
    "print(\"Mean:\")\n",
    "print(np.mean(arr_vals))\n",
    "\n",
    "print(\"Unique values (we'll pass it the array of ones):\")\n",
    "print(np.median(arr_ones))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* So far we've looked at 1D and 2D arrays - you are not limited to these dimensions and can have multiple dimensions as required.\n",
    "* Possible to [scale arrays up as well](https://stackoverflow.com/questions/7525214/how-to-scale-a-numpy-array) e.g. using the Kronecker product (Computes the Kronecker product, a composite array made of blocks of the second array scaled by the first):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([[1, 1],\n",
    "              [0, 1]])\n",
    "n = 2\n",
    "np.kron(a, np.ones((n,n)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Where else does numpy get used?\n",
    "\n",
    "* Image processing e.g. [scikit-image](https://scikit-image.org/)\n",
    "* Pandas (indexing etc.)\n",
    "\n",
    "# More reading/tutorials\n",
    "\n",
    "* Great start (the O'reiley book online): [https://jakevdp.github.io/PythonDataScienceHandbook/02.02-the-basics-of-numpy-arrays.html](https://jakevdp.github.io/PythonDataScienceHandbook/02.02-the-basics-of-numpy-arrays.html)\n",
    "* The numpy reference: [https://docs.scipy.org/doc/numpy/reference/](https://docs.scipy.org/doc/numpy/reference/)\n",
    "* The numpy website: [http://www.numpy.org/](http://www.numpy.org/)\n",
    "* Image manipulation and processing using Numpy and Scipy: [http://scipy-lectures.org/advanced/image_processing/](http://scipy-lectures.org/advanced/image_processing/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}