language/prompts/examples/text_summarization.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2023 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "# Text Summarization with Generative Models on Vertex AI\n",
        "\n",
        "> **NOTE:** This notebook uses the PaLM generative model, which will reach its [discontinuation date in October 2024](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/text#model_versions). Please refer to [this updated notebook](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/prompts/examples/text_summarization.ipynb) for a version which uses the latest Gemini model.\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/language/prompts/examples/text_summarization.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/language/prompts/examples/text_summarization.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/language/prompts/examples/text_summarization.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8e71fd522e56"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Polong Lin](https://github.com/polong-lin) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "Text summarization produces a concise and fluent summary of a longer text document. There are two main text summarization types: extractive and abstractive. Extractive summarization involves selecting critical sentences from the original text and combining them to form a summary. Abstractive summarization involves generating new sentences representing the original text's main points. In this notebook, you go through a few examples of how large language models can help with generating summaries based on text.\n",
        "\n",
        "Learn more about text summarization in the [official documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/text/summarization-prompts)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d975e698c9a4"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you will learn how to use generative models to summarize information from text by working through the following examples:\n",
        "- Transcript summarization\n",
        "- Summarizing text into bullet points\n",
        "- Dialogue summarization with to-dos\n",
        "- Hashtag tokenization\n",
        "- Title & heading generation\n",
        "\n",
        "You also learn how to evaluate model-generated summaries by comparing to human-created summaries using ROUGE as an evaluation framework."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f6d865e68adb"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI Generative AI Studio\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n",
        "and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bs9TZo0GJKCR"
      },
      "source": [
        "## Getting Started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2a5AEr0lkLKD"
      },
      "source": [
        "### Install Vertex AI SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "148dd6321946"
      },
      "outputs": [],
      "source": [
        "!pip install google-cloud-aiplatform --upgrade --user"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FLVWFKFwkLKE"
      },
      "source": [
        "**Colab only:** Uncomment the following cell to restart the kernel or use the button to restart the kernel. For Vertex AI Workbench you can restart the terminal using the button on top. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_Hsqwn4hkLKE"
      },
      "outputs": [],
      "source": [
        "# # Automatically restart kernel after installs so that your environment can access the new packages\n",
        "# import IPython\n",
        "\n",
        "# app = IPython.Application.instance()\n",
        "# app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xe7OuYuGkLKF"
      },
      "source": [
        "### Authenticating your notebook environment\n",
        "* If you are using **Colab** to run this notebook, uncomment the cell below and continue.\n",
        "* If you are using **Vertex AI Workbench**, check out the setup instructions [here](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/setup-env)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "U9Gx2SAZkLKF"
      },
      "outputs": [],
      "source": [
        "# from google.colab import auth\n",
        "# auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "960505627ddf"
      },
      "source": [
        "### Import libraries\n",
        "\n",
        "Let's start by importing the libraries that we will need for this tutorial"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1db41b42c089"
      },
      "source": [
        "**Colab only:** Uncomment the following cell to initialize the Vertex AI SDK. For Vertex AI Workbench, you don't need to run this.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9100a53f80c2"
      },
      "outputs": [],
      "source": [
        "# import vertexai\n",
        "\n",
        "# PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "# vertexai.init(project=PROJECT_ID, location=\"us-central1\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PyQmSRbKA8r-"
      },
      "outputs": [],
      "source": [
        "from vertexai.language_models import TextGenerationModel"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UP76a2la7O-a"
      },
      "source": [
        "### Import models\n",
        "\n",
        "Here we load the pre-trained text generation model called `text-bison`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7isig7e07O-a"
      },
      "outputs": [],
      "source": [
        "generation_model = TextGenerationModel.from_pretrained(\"text-bison\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mu1UAhoTKn51"
      },
      "source": [
        "## Text Summarization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TgZvJeBpJKCS"
      },
      "source": [
        "### Transcript summarization\n",
        "\n",
        "In this first example, you summarize a piece of text on quantum computing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UA2NjngeJKCS"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Provide a very short summary, no more than three sentences, for the following article:\n",
        "\n",
        "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n",
        "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n",
        "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n",
        "To bridge this gap, we will need quantum error correction.\n",
        "Quantum error correction protects information by encoding it across multiple physical qubits to form a 'logical qubit,' and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n",
        "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n",
        "\n",
        "Summary:\n",
        "\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.2, max_output_tokens=1024, top_k=40, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aade04b2e86a"
      },
      "source": [
        "Instead of a summary, we can ask for a TL;DR (\"too long; didn't read\"). You can compare the differences between the outputs generated."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0c0c0f3dfe10"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Provide a TL;DR for the following article:\n",
        "\n",
        "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms. \n",
        "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow. \n",
        "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today. \n",
        "To bridge this gap, we will need quantum error correction. \n",
        "Quantum error correction protects information by encoding it across multiple physical qubits to form a 'logical qubit,' and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations. \n",
        "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n",
        "\n",
        "TL;DR:\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.2, max_output_tokens=1024, top_k=40, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5PATmHivJKCS"
      },
      "source": [
        "### Summarize text into bullet points\n",
        "In the following example, you use same text on quantum computing, but ask the model to summarize it in bullet-point form. Feel free to change the prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2orkDF2VJKCT"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Provide a very short summary in four bullet points for the following article:\n",
        "\n",
        "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n",
        "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n",
        "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n",
        "To bridge this gap, we will need quantum error correction.\n",
        "Quantum error correction protects information by encoding it across multiple physical qubits to form a 'logical qubit,' and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n",
        "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n",
        "\n",
        "Bullet points:\n",
        "\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.2, max_output_tokens=256, top_k=1, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yE7y-clBJKCT"
      },
      "source": [
        "###  Dialogue summarization with to-dos\n",
        "Dialogue summarization involves condensing a conversation into a shorter format so that you don't need to read the whole discussion but can leverage a summary. In this example, you ask the model to summarize an example conversation between an online retail customer and a support agent, and include to-dos at the end."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SV-BWzRhJKCT"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Please generate a summary of the following conversation and at the end summarize the to-do's for the support Agent:\n",
        "\n",
        "Customer: Hi, I'm Larry, and I received the wrong item.\n",
        "\n",
        "Support Agent: Hi, Larry. How would you like to see this resolved?\n",
        "\n",
        "Customer: That's alright. I want to return the item and get a refund, please.\n",
        "\n",
        "Support Agent: Of course. I can process the refund for you now. Can I have your order number, please?\n",
        "\n",
        "Customer: It's [ORDER NUMBER].\n",
        "\n",
        "Support Agent: Thank you. I've processed the refund, and you will receive your money back within 14 days.\n",
        "\n",
        "Customer: Thank you very much.\n",
        "\n",
        "Support Agent: You're welcome, Larry. Have a good day!\n",
        "\n",
        "Summary:\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.2, max_output_tokens=256, top_k=40, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LlOgWzmNJKCT"
      },
      "source": [
        "###  Hashtag tokenization\n",
        "Hashtag tokenization is the process of taking a piece of text and getting the hashtag \"tokens\" out of it. You can use this, for example, if you want to generate hashtags for your social media campaigns. In this example, you take [this tweet from Google Cloud](https://twitter.com/googlecloud/status/1649127992348606469) and generate some hashtags you can use."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BWa8rNV0JKCT"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Tokenize the hashtags of this tweet:\n",
        "\n",
        "Google Cloud\n",
        "@googlecloud\n",
        "How can data help our changing planet? 🌎\n",
        "\n",
        "In honor of #EarthDay this weekend, we're proud to share how we're partnering with\n",
        "@ClimateEngine\n",
        " to harness the power of geospatial data and drive toward a more sustainable future.\n",
        "\n",
        "Check out how → https://goo.gle/3mOUfts\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.8, max_output_tokens=1024, top_k=40, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7f-w7mUxJKCT"
      },
      "source": [
        "### Title & heading generation\n",
        "Below, you ask the model to generate five options for possible title/heading combos for a given piece of text."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MWNri4DTJKCU"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Write a title for this text, give me five options:\n",
        "Whether helping physicians identify disease or finding photos of 'hugs,' AI is behind a lot of the work we do at Google. And at our Arts & Culture Lab in Paris, we've been experimenting with how AI can be used for the benefit of culture.\n",
        "Today, we're sharing our latest experiments—prototypes that build on seven years of work in partnership the 1,500 cultural institutions around the world.\n",
        "Each of these experimental applications runs AI algorithms in the background to let you unearth cultural connections hidden in archives—and even find artworks that match your home decor.\"\n",
        "\"\"\"\n",
        "\n",
        "print(\n",
        "    generation_model.predict(\n",
        "        prompt, temperature=0.8, max_output_tokens=256, top_k=1, top_p=0.8\n",
        "    ).text\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zcpmZnwKJKCU"
      },
      "source": [
        "## Evaluation\n",
        "You can evaluate the outputs from summarization tasks using [ROUGE](https://en.wikipedia.org/wiki/ROUGE_(metric)) as an evaluation framework. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) are measures to automatically determine the quality of a summary by comparing it to other (ideal) summaries created by humans. The measures count the number of overlapping units such as n-gram, word sequences, and word pairs between the computer-generated summary to be evaluated and the ideal summaries created by humans.\n",
        "\n",
        "\n",
        "The first step is to install the ROUGE library."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tJcl38ElJKCU"
      },
      "outputs": [],
      "source": [
        "!pip install rouge"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iD9eKq3SJKCU"
      },
      "source": [
        "Create a summary from a language model that you can use to compare against a human-generated summary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "37m_fb-HJKCU"
      },
      "outputs": [],
      "source": [
        "from rouge import Rouge\n",
        "\n",
        "ROUGE = Rouge()\n",
        "\n",
        "prompt = \"\"\"\n",
        "Provide a very short, maximum four sentences, summary for the following article:\n",
        "\n",
        "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n",
        "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n",
        "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n",
        "To bridge this gap, we will need quantum error correction.\n",
        "Quantum error correction protects information by encoding it across multiple physical qubits to form a 'logical qubit,' and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n",
        "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n",
        "\n",
        "Summary:\n",
        "\n",
        "\"\"\"\n",
        "\n",
        "candidate = generation_model.predict(\n",
        "    prompt, temperature=0.1, max_output_tokens=1024, top_k=40, top_p=0.9\n",
        ").text\n",
        "\n",
        "print(candidate)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b44f9872e1ba"
      },
      "source": [
        "You will also need a human-generated summary that we will use to compare to the `candidate` generated by the model. We will call this `reference`. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D0qNdPzOJKCc"
      },
      "outputs": [],
      "source": [
        "reference = \"Quantum computers are sensitive to noise and errors. To bridge this gap, we will need quantum error correction. Quantum error correction protects information by encoding across multiple physical qubits to form a 'logical qubit'.\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1KKaYhzwJKCc"
      },
      "source": [
        "Now you can take the candidate and reference to evaluate the performance. In this case, ROUGE will give you:\n",
        "\n",
        "- `rouge-1`, which measures unigram overlap\n",
        "- `rouge-2`, which measures bigram overlap\n",
        "- `rouge-l`, which measures the longest common subsequence"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FHUH6VuTJKCc"
      },
      "outputs": [],
      "source": [
        "ROUGE.get_scores(candidate, reference)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "text_summarization.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}