9 Web API.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Web APIs\n",
    "\n",
    "## 1. Consuming Web APIs\n",
    "\n",
    "(Partially adapted from https://www.dataquest.io/blog/python-api-tutorial/)\n",
    "\n",
    "* https://randomuser.me/ is a random user generator \n",
    "*\tIt has an API. Go to https://api.randomuser.me/ with your browser\n",
    "*\tYou get a JSON (JavaScript Object Notation) back. A JSON is similar to a Python dictionary.  \n",
    "*  Refresh the browser\n",
    "*\tYou can add query parameters. They are added to the URL with a ? . You then add then the parameter name = the value. More the one parameter are connected with a & e.g.\n",
    "\thttp://api.open-notify.org/iss-pass.json?lat=37.78&lon=-122.41\n",
    "\n",
    "*\t Go the documentation https://randomuser.me/documentation and check how you can get multiple users and to specify constraints on the output\n",
    "*\tGo to your browser and add parameters to https://api.randomuser.me/ so that you get 5 results of only males from the US\n",
    "\n",
    "Enter the URL of your solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO: get 5 results of only males from the US\n",
    "# Enter the URL of your solution:\n",
    "url = None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* You can also get the data from the command line. Open the command line and write\n",
    "\n",
    "```bash\n",
    "curl -s https://api.randomuser.me/\n",
    "```\n",
    "\n",
    "You can also run Bash commands directly in your Jupyter Notebook with !:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!curl -s https://api.randomuser.me/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Import the two libraries \n",
    "    * `requests` and \n",
    "    * `json`. \n",
    "\n",
    "You can find the documentation for the requests package here:\n",
    " http://www.python-requests.org/en/latest/ \n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* With the requests package you can call a Web API with the URL and the method get"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "response = requests.get(\"https://api.randomuser.me/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Print the status code of the request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(response.status_code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The meaning of the status codes are:**\n",
    "* 200: everything went okay, and the result has been returned (if any)\n",
    "* 301: the server is redirecting you to a different endpoint. This can happen when a company switches domain names, or an endpoint name is changed.\n",
    "* 401: the server thinks you're not authenticated. This happens when you don't send the right credentials to access an API.\n",
    "* 400: the server thinks you made a bad request. This can happen when you don't send along the right data, among other things.\n",
    "* 403: the resource you're trying to access is forbidden – you don't have the right permissions to see it.\n",
    "* 404: the resource you tried to access wasn't found on the server.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can specify the query parameters for a URL with a Python dictionary like this:\n",
    "```python\n",
    "parameters = {\"lat\": 37.78, \"lon\": -122.41}\n",
    "```\n",
    "\n",
    "And pass the parameter to the request like this\n",
    "```python\n",
    "response = requests.get(\"http://api.open-notify.org/iss-pass.json\", params=parameters)\n",
    "```\n",
    "This is the same as \n",
    "```python\n",
    "response = requests.get(\"http://api.open-notify.org/iss-pass.json?lat=37.78&lon=-122.41\")\n",
    "```\n",
    "\n",
    "Alternatively you could build also the URL with the parameters by yourself with string concatenation. \n",
    "\n",
    "* Get with the request method 10 results of only males from the US.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO: Get with the request method 10 results of only males from the US."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* You can show the result of the request with the method text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "response.text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* You can convert the data from JSON to a Python dictionary with the package JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = json.loads(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Check the type of variable data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* *pretty-print* (pprint) prints complex data structures like dictionary prettier.  https://docs.python.org/3/library/pprint.html "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pprint import pprint\n",
    "pprint(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Loop through the dictionary and print all first names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Get all astronauts who are right now in space. You get the information about the Web APU from here  http://open-notify.org/Open-Notify-API/People-In-Space/ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Print the number of people that are right now in space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Print the names of all astronauts "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* A lot of Web APIs require a api-key for interacting with them (like Twitter, Facebook, …). You find at http://www.python-requests.org/en/latest/user/authentication/ more information for Authentication for Web APIs with the request package\n",
    "* There are also special Python packages for interacting with services. E.g. for Twitter: http://www.tweepy.org/ or  https://github.com/bear/python-twitter \n",
    "\n",
    "See e.g. http://socialmedia-class.org/twittertutorial.html for a tutorial"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Creating a Web API\n",
    "\n",
    "* Create a folder `webapi` and change into it.\n",
    "* Create in the `webapi` folder a file with the name `Dockerfile` with the following content:\n",
    "\n",
    "----\n",
    "```bash\n",
    "# Use an official Python runtime as a parent image\n",
    "FROM python:3.10-slim-buster\n",
    "\n",
    "# Set the working directory to /app\n",
    "WORKDIR /app\n",
    "\n",
    "# Copy requirements.txt into the container at /app\n",
    "COPY app/requirements.txt requirements.txt\n",
    "\n",
    "# Install any needed packages specified in requirements.txt\n",
    "RUN pip install -r requirements.txt\n",
    "\n",
    "# Copy the current directory contents into the container at /app\n",
    "COPY app/ /app\n",
    "\n",
    "# Make port 80 available to the world outside this container\n",
    "EXPOSE 80\n",
    "\n",
    "# Run app.py when the container launches\n",
    "CMD [\"python\", \"app.py\"]\n",
    "```\n",
    "\n",
    "-----\n",
    "\n",
    "* We can also use Docker compose with just one service. Create in your `webapi` folder a `docker-compose.yml` file:\n",
    "\n",
    "-----\n",
    "\n",
    "```yaml\n",
    "version: '3'\n",
    "services:\n",
    "  api:\n",
    "    build: .\n",
    "    ports:\n",
    "      - \"5000:80\"\n",
    "    restart: always\n",
    "    volumes:\n",
    "      - ./app:/app\n",
    "```\n",
    "-----\n",
    "\n",
    "* Create a folder in the `webapi` folder a new folder with the name `app`\n",
    "* We will build a web API with `Flask` (http://flask.pocoo.org/) . Create a `requirements.txt` file in the `app` folder. Here we can specify all python `pip` packages that we need:\n",
    "\n",
    "-----\n",
    "```bash\n",
    "Flask\n",
    "```\n",
    "-----\n",
    "\n",
    "* Create the `app.py` file in the `app` folder:\n",
    "\n",
    "-----\n",
    "```python\n",
    "from flask import Flask\n",
    "from flask import request, jsonify\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "courses = [\n",
    "    {'id': 0,\n",
    "     'title': 'Data Science',\n",
    "     'professor': 'Markus Löcher',\n",
    "     'semester': '1'},\n",
    "    {'id': 1,\n",
    "     'title': 'Data Warehousing',\n",
    "     'professor': 'Roland M. Mueller',\n",
    "     'semester': '1'},\n",
    "    {'id': 2,\n",
    "     'title': 'Business Process Management',\n",
    "     'professor': 'Frank Habermann',\n",
    "     'semester': '1'},\n",
    "    {'id': 3,\n",
    "     'title': 'Complexity and Organizational Decision-Making',\n",
    "     'professor': 'Frank Habermann',\n",
    "     'semester': '1'},\n",
    "    {'id': 4,\n",
    "     'title': 'Text, Web and Social Media Analytics Lab',\n",
    "     'professor': 'Diana Hristova',\n",
    "     'semester': '2'},\n",
    "    {'id': 5,\n",
    "     'title': 'Enterprise Architectures for Big Data',\n",
    "     'professor': 'Roland M. Mueller',\n",
    "     'semester': '2'},\n",
    "    {'id': 6,\n",
    "     'title': 'Business Process Innovation Lab',\n",
    "     'professor': 'Frank Habermann',\n",
    "     'semester': '2'},\n",
    "    {'id': 7,\n",
    "     'title': 'IT-Security and Privacy',\n",
    "     'professor': 'Dennis Uckel',\n",
    "     'semester': '2'},\n",
    "    {'id': 8,\n",
    "     'title': 'Research Methods',\n",
    "     'professor': 'Roland M. Mueller',\n",
    "     'semester': '2'},\n",
    "]\n",
    "\n",
    "@app.route('/api/v1/courses/all', methods=['GET'])\n",
    "def api_all():\n",
    "    return jsonify(courses)\n",
    "\n",
    "@app.route('/api/v1/courses', methods=['GET'])\n",
    "def api_id():\n",
    "    # Check if an ID was provided as part of the URL.\n",
    "    # If ID is provided, assign it to a variable.\n",
    "    # If no ID is provided, display an error in the browser.\n",
    "    if 'id' in request.args:\n",
    "        id = int(request.args['id'])\n",
    "    else:\n",
    "        return \"Error: No id field provided. Please specify an id.\"\n",
    "\n",
    "    # Create an empty list for our results\n",
    "    results = []\n",
    "\n",
    "    # Loop through the data and match results that fit the requested ID.\n",
    "    # IDs are unique, but other fields might return many results\n",
    "    for course in courses:\n",
    "        if course['id'] == id:\n",
    "            results.append(course)\n",
    "\n",
    "    # Use the jsonify function from Flask to convert our list of\n",
    "    # Python dictionaries to the JSON format.\n",
    "    return jsonify(results)\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    app.run(host=\"0.0.0.0\", port=80, debug=True)\n",
    "```\n",
    "-----\n",
    "* Open http://localhost:5000/api/v1/courses/all in a browser\n",
    "* Open http://localhost:5000/api/v1/courses?id=5 in a browser\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "* Use your own API here in the Jupyter Notebook with Python and print all names of all courses "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Add the possibility to find courses based on the semester\n",
    "* Use your API in Python and print all names of all courses in the second semester"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Add a function that can convert Fahrenheit to Celsius \n",
    "* Call your API and get the Celsius value for 100°F Fahrenheit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TODO"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}