Working-with-Text-Files.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Working with Text Files\n",
    "In this section we'll cover\n",
    " * Working with f-strings (formatted string literals) to format printed text\n",
    " * Working with Files - opening, reading, writing and appending text files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Formatted String Literals (f-strings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Introduced in Python 3.6, <strong>f-strings</strong> offer several benefits over the older `.format()` string method. <br>For one, you can bring outside variables immediately into to the string rather than pass them through as keyword arguments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "His name is Fred.\n",
      "His name is Fred.\n"
     ]
    }
   ],
   "source": [
    "name = 'Fred'\n",
    "\n",
    "# Using the old .format() method:\n",
    "print('His name is {var}.'.format(var=name))\n",
    "\n",
    "# Using f-strings:\n",
    "print(f'His name is {name}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pass `!r` to get the <strong>string representation</strong>:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "His name is 'Fred'\n"
     ]
    }
   ],
   "source": [
    "print(f'His name is {name!r}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Be careful not to let quotation marks in the replacement fields conflict with the quoting used in the outer string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (3751357111.py, line 3)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"/var/folders/14/nc2ygsds1flfyq_lf5n38g600000gn/T/ipykernel_2237/3751357111.py\"\u001b[0;36m, line \u001b[0;32m3\u001b[0m\n\u001b[0;31m    print(f'Address: {d['a']} Main Street')\u001b[0m\n\u001b[0m                         ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "d = {'a':123,'b':456}\n",
    "\n",
    "print(f'Address: {d['a']} Main Street')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead, use different styles of quotation marks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Address: 123 Main Street\n"
     ]
    }
   ],
   "source": [
    "d = {'a':123,'b':456}\n",
    "\n",
    "print(f\"Address: {d['a']} Main Street\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Minimum Widths, Alignment and Padding\n",
    "You can pass arguments inside a nested set of curly braces to set a minimum width for the field, the alignment and even padding characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Author     Topic    Pages  \n",
      "Twain      Rafting      601\n",
      "Feynman    Physics       95\n",
      "Hamilton   Mythology     144\n"
     ]
    }
   ],
   "source": [
    "library = [('Author', 'Topic', 'Pages'), ('Twain', 'Rafting', 601), ('Feynman', 'Physics', 95), ('Hamilton', 'Mythology', 144)]\n",
    "\n",
    "for book in library:\n",
    "    print(f'{book[0]:{10}} {book[1]:{8}} {book[2]:{7}}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the first three lines align, except `Pages` follows a default left-alignment while numbers are right-aligned. Also, the fourth line's page number is pushed to the right as `Mythology` exceeds the minimum field width of `8`. When setting minimum field widths make sure to take the longest item into account.\n",
    "\n",
    "To set the alignment, use the character `<` for left-align,  `^` for center, `>` for right.<br>\n",
    "To set padding, precede the alignment character with the padding character (`-` and `.` are common choices).\n",
    "\n",
    "Let's make some adjustments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Author     Topic      ..Pages\n",
      "Twain      Rafting    ....601\n",
      "Feynman    Physics    .....95\n",
      "Hamilton   Mythology  ....144\n"
     ]
    }
   ],
   "source": [
    "for book in library:\n",
    "    print(f'{book[0]:{10}} {book[1]:{10}} {book[2]:.>{7}}') # here .> was added"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Date Formatting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "January 27, 2018\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "today = datetime(year=2018, month=1, day=27)\n",
    "\n",
    "print(f'{today:%B %d, %Y}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Files\n",
    "\n",
    "Python uses file objects to interact with external files on your computer. These file objects can be any sort of file you have on your computer, whether it be an audio file, a text file, emails, Excel documents, etc. Note: You will probably need to install certain libraries or modules to interact with those various file types, but they are easily available. (We will cover downloading modules later on in the course).\n",
    "\n",
    "Python has a built-in open function that allows us to open and play with basic file types. First we will need a file though. We're going to use some IPython magic to create a text file!\n",
    "\n",
    "## Creating a File with IPython\n",
    "#### This function is specific to jupyter notebooks! Alternatively, quickly create a simple .txt file with Sublime text editor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting test.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile test.txt\n",
    "Hello, this is a quick test file.\n",
    "This is the second line of the file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Opening a File\n",
    "\n",
    "### Know Your File's Location\n",
    "\n",
    "It's easy to get an error on this step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "myfile = open('test.txt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To avoid this error, make sure your .txt file is saved in the same location as your notebook. To check your notebook location, use **pwd**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Mike\\\\NLP-Bootcamp\\\\00-Python-Text-Basics'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Alternatively, to grab files from any location on your computer, simply pass in the entire file path. **\n",
    "\n",
    "For Windows you need to use double \\ so python doesn't treat the second \\ as an escape character, a file path is in the form:\n",
    "\n",
    "    myfile = open(\"C:\\\\Users\\\\YourUserName\\\\Home\\\\Folder\\\\myfile.txt\")\n",
    "\n",
    "For MacOS and Linux you use slashes in the opposite direction:\n",
    "\n",
    "    myfile = open(\"/Users/YourUserName/Folder/myfile.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Open the text.txt file we created earlier\n",
    "my_file = open('test.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<_io.TextIOWrapper name='test.txt' mode='r' encoding='cp1252'>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`my_file` is now an open file object held in memory. We'll perform some reading and writing exercises, and then we have to close the file to free up memory.\n",
    "\n",
    "### .read() and .seek()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, this is a quick test file.\\nThis is the second line of the file.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can now read the file\n",
    "my_file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# But what happens if we try to read it again?\n",
    "my_file.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This happens because you can imagine the reading \"cursor\" is at the end of the file after having read it. So there is nothing left to read. We can reset the \"cursor\" like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Seek to the start of file (index 0)\n",
    "my_file.seek(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello, this is a quick test file.\\nThis is the second line of the file.'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now read again\n",
    "my_file.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .readlines()\n",
    "You can read a file line by line using the readlines method. Use caution with large files, since everything will be held in memory. We will learn how to iterate over large files later in the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Hello, this is a quick test file.\\n', 'This is the second line of the file.']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Readlines returns a list of the lines in the file\n",
    "my_file.seek(0)\n",
    "my_file.readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you have finished using a file, it is always good practice to close it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing to a File\n",
    "\n",
    "By default, the `open()` function will only allow us to read the file. We need to pass the argument `'w'` to write over the file. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a second argument to the function, 'w' which stands for write.\n",
    "# Passing 'w+' lets us read and write to the file\n",
    "\n",
    "my_file = open('test.txt','w+')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger\" style=\"margin: 20px\">**Use caution!**<br>\n",
    "Opening a file with 'w' or 'w+' *truncates the original*, meaning that anything that was in the original file **is deleted**!</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Write to the file\n",
    "my_file.write('This is a new first line')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This is a new first line'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the file\n",
    "my_file.seek(0)\n",
    "my_file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_file.close()  # always do this when you're done with a file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Appending to a File\n",
    "Passing the argument `'a'` opens the file and puts the pointer at the end, so anything written is appended. Like `'w+'`, `'a+'` lets us read and write to a file. If the file does not exist, one will be created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "23"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_file = open('test.txt','a+')\n",
    "my_file.write('\\nThis line is being appended to test.txt')\n",
    "my_file.write('\\nAnd another line here.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a new first line\n",
      "This line is being appended to test.txt\n",
      "And another line here.\n"
     ]
    }
   ],
   "source": [
    "my_file.seek(0)\n",
    "print(my_file.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Appending with `%%writefile`\n",
    "Jupyter notebook users can do the same thing using IPython cell magic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Appending to test.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile -a test.txt\n",
    "\n",
    "This is more text being appended to test.txt\n",
    "And another line here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add a blank space if you want the first line to begin on its own line, as Jupyter won't recognize escape sequences like `\\n`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aliases and Context Managers\n",
    "You can assign temporary variable names as aliases, and manage the opening and closing of files automatically using a context manager:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a new first line\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('test.txt','r') as txt:\n",
    "    first_line = txt.readlines()[0]\n",
    "    \n",
    "print(first_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the `with ... as ...:` context manager automatically closed `test.txt` after assigning the first line of text to first_line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "I/O operation on closed file.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-28-39ca4397fa0a>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mtxt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mread\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m: I/O operation on closed file."
     ]
    }
   ],
   "source": [
    "txt.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterating through a File"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a new first line\n",
      "This line is being appended to test.txt\n",
      "And another line here.\n",
      "This is more text being appended to test.txt\n",
      "And another line here."
     ]
    }
   ],
   "source": [
    "with open('test.txt','r') as txt:\n",
    "    for line in txt:\n",
    "        print(line, end='')  # the end='' argument removes extra linebreaks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Now you should be familiar with formatted string literals and working with text files.\n",
    "## Next up: Working with PDF Text"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}