t81_559_class_04_2_memory_buffer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "whjsJasuhstV"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_04_2_memory_buffer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "euOZxlIMhstX"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 4: LangChain: Chat and Memory**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d4Yov72PhstY"
   },
   "source": [
    "# Module 4 Material\n",
    "\n",
    "* Part 4.1: LangChain Conversations [[Video]](https://www.youtube.com/watch?v=Effbhxq07Ag) [[Notebook]](t81_559_class_04_1_langchain_chat.ipynb)\n",
    "* **Part 4.2: Conversation Buffer Window Memory** [[Video]](https://www.youtube.com/watch?v=14RgiFVGfAA) [[Notebook]](t81_559_class_04_2_memory_buffer.ipynb)\n",
    "* Part 4.3: Conversation Token Buffer Memory [[Video]](https://www.youtube.com/watch?v=QTe5g2c3bSM) [[Notebook]](t81_559_class_04_3_memory_token.ipynb)\n",
    "* Part 4.4: Conversation Summary Memory [[Video]](https://www.youtube.com/watch?v=asZQ8Ktqmt8) [[Notebook]](t81_559_class_04_4_memory_summary.ipynb)\n",
    "* Part 4.5: Persisting Langchain Memory [[Video]](https://www.youtube.com/watch?v=sjCyqqOQcPA) [[Notebook]](t81_559_class_04_5_memory_persist.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AcAUP0c3hstY"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running and maps Google Drive if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xsI496h5hstZ",
    "outputId": "ad8b02ba-a203-41c3-a27a-1db0cf2e4ced"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n",
      "Collecting langchain\n",
      "  Downloading langchain-0.1.17-py3-none-any.whl (867 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m867.6/867.6 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain_openai\n",
      "  Downloading langchain_openai-0.1.5-py3-none-any.whl (34 kB)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.29)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.5)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
      "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
      "  Downloading dataclasses_json-0.6.5-py3-none-any.whl (28 kB)\n",
      "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
      "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
      "Collecting langchain-community<0.1,>=0.0.36 (from langchain)\n",
      "  Downloading langchain_community-0.0.36-py3-none-any.whl (2.0 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m19.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-core<0.2.0,>=0.1.48 (from langchain)\n",
      "  Downloading langchain_core-0.1.48-py3-none-any.whl (302 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.9/302.9 kB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-text-splitters<0.1,>=0.0.1 (from langchain)\n",
      "  Downloading langchain_text_splitters-0.0.1-py3-none-any.whl (21 kB)\n",
      "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
      "  Downloading langsmith-0.1.52-py3-none-any.whl (116 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.4/116.4 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.7.1)\n",
      "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
      "Collecting openai<2.0.0,>=1.10.0 (from langchain_openai)\n",
      "  Downloading openai-1.25.0-py3-none-any.whl (312 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m312.9/312.9 kB\u001b[0m \u001b[31m10.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting tiktoken<1,>=0.5.2 (from langchain_openai)\n",
      "  Downloading tiktoken-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m16.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
      "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading marshmallow-3.21.2-py3-none-any.whl (49 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.3/49.3 kB\u001b[0m \u001b[31m2.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
      "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
      "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
      "Collecting packaging<24.0,>=23.2 (from langchain-core<0.2.0,>=0.1.48->langchain)\n",
      "  Downloading packaging-23.2-py3-none-any.whl (53 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
      "  Downloading orjson-3.10.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m142.7/142.7 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (3.7.1)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.7.0)\n",
      "Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.3.1)\n",
      "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.66.2)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.11.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.18.2 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.18.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
      "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<1,>=0.5.2->langchain_openai) (2023.12.25)\n",
      "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.10.0->langchain_openai) (1.2.1)\n",
      "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
      "Installing collected packages: packaging, orjson, mypy-extensions, jsonpointer, h11, typing-inspect, tiktoken, marshmallow, jsonpatch, httpcore, langsmith, httpx, dataclasses-json, openai, langchain-core, langchain-text-splitters, langchain_openai, langchain-community, langchain\n",
      "  Attempting uninstall: packaging\n",
      "    Found existing installation: packaging 24.0\n",
      "    Uninstalling packaging-24.0:\n",
      "      Successfully uninstalled packaging-24.0\n",
      "Successfully installed dataclasses-json-0.6.5 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.17 langchain-community-0.0.36 langchain-core-0.1.48 langchain-text-splitters-0.0.1 langchain_openai-0.1.5 langsmith-0.1.52 marshmallow-3.21.2 mypy-extensions-1.0.0 openai-1.25.0 orjson-3.10.2 packaging-23.2 tiktoken-0.6.0 typing-inspect-0.9.0\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "try:\n",
    "    from google.colab import drive, userdata\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False\n",
    "\n",
    "# OpenAI Secrets\n",
    "if COLAB:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
    "\n",
    "# Install needed libraries in CoLab\n",
    "if COLAB:\n",
    "    !pip install langchain langchain_openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pC9A-LaYhsta"
   },
   "source": [
    "# 4.2: Conversation Buffer Window Memory\n",
    "\n",
    "We previously saw that we could build up an LLM chat client by building an ever-increasing script of what the human and AI said in the conversation. We constantly add the human response and wait to see what the AI will respond to next. This cycle continues as long as the chat.\n",
    "\n",
    "This ever-increasing chat memory is a typical pattern for LLMs, and as a result, LangChain has several predefined Python objects that allow you to implement this sort of memory-based chatbot.\n",
    "\n",
    "## Creating a Chat Conversation\n",
    "\n",
    "ConversationChain in LangChain is a framework designed to facilitate the development and management of conversational AI systems. It primarily orchestrates the interaction between different components, such as the dialogue management system, language models, and various types of memory that retain information from the conversation to improve response relevance and coherence.\n",
    "\n",
    "In this context, memory types play a crucial role. They help the system remember and utilize past interactions to maintain context and enhance the continuity of the conversation. This system frees us from tracking the conversation as performed manually in the pervious section.\n",
    "\n",
    "We first examine ConversationBufferWindowMemory is a dynamic memory model designed to manage the flow of a dialogue by retaining a record of the last K interactions within a conversation. This method ensures that the memory buffer maintains a manageable size, avoiding overflow and performance issues that can arise with an excessively large interaction history. By focusing on the most recent interactions, this memory type effectively creates a \"sliding window\" that continuously updates as new exchanges occur, allowing for contextually relevant responses while efficiently managing system resources. This approach is particularly beneficial in applications where maintaining an immediate and context-aware dialogue is crucial, such as in customer service bots or interactive learning tools.\n",
    "\n",
    "The provided code snippet demonstrates how to set up and use a conversational AI system using the LangChain framework, specifically focusing on maintaining a concise memory of recent interactions. The ConversationChain class orchestrates the conversation flow, integrating various components such as language model, memory, and prompt formatting.\n",
    "\n",
    "Firstly, several modules are imported, including ConversationChain for managing the conversation, ConversationBufferWindowMemory for handling the memory of recent interactions, and ChatOpenAI to interface with OpenAI's language models. The PromptTemplate is used to define the structure of the prompts sent to the language model.\n",
    "\n",
    "In the setup, MODEL specifies the particular version of the language model (gpt-4o-mini), and TEMPLATE outlines the format of the conversation, integrating previous dialogue history and the current user input into the prompt. This structured prompt is then used to create a PromptTemplate object.\n",
    "\n",
    "The begin_conversation function initializes the language model and sets up the memory to store the last five interactions (k=5). This sliding window of memory helps keep the conversation relevant and efficient by focusing on the most recent exchanges. The ConversationChain object is created with the prompt template, the language model, and the memory buffer, and it's ready to process conversation inputs.\n",
    "\n",
    "The converse function takes a conversation object and a user prompt to produce responses. It uses the conversation object's invoke method, which applies the prompt template, consults the memory, and generates a response based on the current and recent dialogue context.\n",
    "\n",
    "Overall, this setup is optimized for creating a responsive and context-aware conversational AI that doesn't overload with too much past information, maintaining relevancy and efficiency in ongoing interactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TMF-rtxgRAea"
   },
   "outputs": [],
   "source": [
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferWindowMemory\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts.chat import PromptTemplate\n",
    "from IPython.display import display_markdown\n",
    "\n",
    "MODEL = 'gpt-4o-mini'\n",
    "TEMPLATE = \"\"\"You are a helpful assistant. Format answers with markdown.\n",
    "\n",
    "Current conversation:\n",
    "{history}\n",
    "Human: {input}\n",
    "AI:\"\"\"\n",
    "PROMPT_TEMPLATE = PromptTemplate(input_variables=[\"history\", \"input\"], template=TEMPLATE)\n",
    "\n",
    "def begin_conversation():\n",
    "    # Initialize the OpenAI LLM with your API key\n",
    "    llm = ChatOpenAI(\n",
    "        model=MODEL,\n",
    "        temperature=0.0,\n",
    "        n=1\n",
    "    )\n",
    "\n",
    "    # Initialize memory and conversation\n",
    "    memory = ConversationBufferWindowMemory(k=5)\n",
    "    conversation = ConversationChain(\n",
    "        prompt=PROMPT_TEMPLATE,\n",
    "        llm=llm,\n",
    "        memory=memory,\n",
    "        verbose=False\n",
    "    )\n",
    "\n",
    "    return conversation\n",
    "\n",
    "def converse(conversation, prompt):\n",
    "    output = conversation.invoke(prompt)\n",
    "    return output['response']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ClPhLkGldhPt"
   },
   "source": [
    "We can now carry on a simple conversation with the LLM, using LangChain to track the conversation memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 93
    },
    "id": "ydaqwgRiH4D6",
    "outputId": "b9c6d85c-918f-48a8-bf61-dd49837d0337"
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "I'm sorry, but I don't have access to your personal information."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Nice to meet you, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "I'm sorry, but I don't have access to your personal information."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "MODEL = 'gpt-4o-mini'\n",
    "\n",
    "# Initialize the OpenAI LLM with your API key\n",
    "llm = ChatOpenAI(\n",
    "  model=MODEL,\n",
    "  temperature= 0.3,\n",
    "  n= 1)\n",
    "\n",
    "conversation = begin_conversation()\n",
    "output = converse(conversation, \"Hello, what is my name?\")\n",
    "display_markdown(output,raw=True)\n",
    "output = converse(conversation, \"Oh sorry, my name is Jeff.\")\n",
    "display_markdown(output,raw=True)\n",
    "output = converse(conversation, \"What is my name?\")\n",
    "display_markdown(output,raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ulYUrfVcnlgF"
   },
   "source": [
    "## Conversing with the LLM in Markdown\n",
    "\n",
    "Just as before, we can request that the LLM output be in mardown. This allows code and tables to be represented clearly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MHL6Ik8IM2PA"
   },
   "outputs": [],
   "source": [
    "def chat(conversation, prompt):\n",
    "  print(f\"Human: {prompt}\")\n",
    "  output = converse(conversation, prompt)\n",
    "  display_markdown(output,raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FvzsYl3Qplac"
   },
   "source": [
    "The provided code sequence demonstrates a conversation between a human user and a Large Language Model (LLM), making use of the chat function to interactively manage the conversation and display responses in Markdown format. This approach allows for a dynamic and contextually aware chat, while also enhancing the visual and structural presentation of the responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 366
    },
    "id": "PtLDak7TM_FU",
    "outputId": "74b1a787-f0bd-4add-c481-6bfe5603cb96"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm sorry, but I don't have access to that information."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Okay, then let me introduce myself, my name is Jeff\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Nice to meet you, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Your name is Jeff. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Give me a table of the 5 most populus cities with population and country.\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "| City         | Population | Country |\n",
       "|--------------|------------|---------|\n",
       "| Tokyo        | 37,833,000 | Japan   |\n",
       "| Delhi        | 30,291,000 | India   |\n",
       "| Shanghai     | 27,058,000 | China   |\n",
       "| Sao Paulo    | 22,043,000 | Brazil  |\n",
       "| Mexico City  | 21,782,000 | Mexico  |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "conversation = begin_conversation()\n",
    "chat(conversation, \"What is my name?\")\n",
    "chat(conversation, \"Okay, then let me introduce myself, my name is Jeff\")\n",
    "chat(conversation, \"What is my name?\")\n",
    "chat(conversation, \"Give me a table of the 5 most populus cities with population and country.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GeY-9YSOno_t"
   },
   "source": [
    "## Constraining the Conversation with a System Prompt\n",
    "\n",
    "You can use the system prompt to constrain the conversation to a specific topic. Here, we provide a simple agent that will only discuss life insurance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "BoGPv80xG3Ss"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 288
    },
    "id": "NyGpLJmWjGNq",
    "outputId": "9d2b3222-0ca4-434f-ab78-56f928bea144"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Okay, then let me introduce myself, my name is Jeff\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is your favorite programming language?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm here to help answer questions about life insurance. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is the difference between a term and whole life policy?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "A term life insurance policy provides coverage for a specific period of time, such as 10, 20, or 30 years. It is typically more affordable and offers a death benefit if the insured passes away during the term. A whole life insurance policy, on the other hand, provides coverage for the entire lifetime of the insured. It also includes a cash value component that grows over time and can be borrowed against or withdrawn. Whole life insurance premiums are usually higher than term life premiums."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferWindowMemory\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts.chat import PromptTemplate\n",
    "from IPython.display import display_markdown\n",
    "\n",
    "MODEL = 'gpt-4o-mini'\n",
    "\n",
    "\n",
    "def begin_conversation_insurance():\n",
    "    TEMPLATE = \"\"\"You are a helpful agent to answer questions about life insurance. Do not talk\n",
    "    about anything else with users. . Format answers with markdown.\n",
    "\n",
    "    Current conversation:\n",
    "    {history}\n",
    "    Human: {input}\n",
    "    AI:\"\"\"\n",
    "    PROMPT_TEMPLATE = PromptTemplate(input_variables=[\"history\", \"input\"], template=TEMPLATE)\n",
    "\n",
    "    # Initialize the OpenAI LLM with your API key\n",
    "    llm = ChatOpenAI(\n",
    "        model=MODEL,\n",
    "        temperature=0.0,\n",
    "        n=1\n",
    "    )\n",
    "\n",
    "    # Initialize memory and conversation\n",
    "    memory = ConversationBufferWindowMemory(k=5)\n",
    "    conversation = ConversationChain(\n",
    "        prompt=PROMPT_TEMPLATE,\n",
    "        llm=llm,\n",
    "        memory=memory,\n",
    "        verbose=False\n",
    "    )\n",
    "\n",
    "    return conversation\n",
    "\n",
    "conversation = begin_conversation_insurance()\n",
    "chat(conversation, \"What is my name?\")\n",
    "chat(conversation, \"Okay, then let me introduce myself, my name is Jeff\")\n",
    "chat(conversation, \"What is my name?\")\n",
    "chat(conversation, \"What is your favorite programming language?\")\n",
    "chat(conversation, \"What is the difference between a term and whole life policy?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qgwRMwnlH5Ou"
   },
   "source": [
    "##Examining the Conversation Memory\n",
    "\n",
    "We can quickly look inside the memory of the LangChain-managed chat memory and see our conversation memory with the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WZDN67SOHkNo",
    "outputId": "fe76bbba-c618-4773-94be-f1bea6a0b78d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: What is my name?\n",
      "AI: I'm here to help answer questions about life insurance. How can I assist you today?\n",
      "Human: Okay, then let me introduce myself, my name is Jeff\n",
      "AI: I'm here to help answer questions about life insurance. How can I assist you today?\n",
      "Human: What is my name?\n",
      "AI: I'm here to help answer questions about life insurance. How can I assist you today?\n",
      "Human: What is your favorite programming language?\n",
      "AI: I'm here to help answer questions about life insurance. How can I assist you today?\n",
      "Human: What is the difference between a term and whole life policy?\n",
      "AI: A term life insurance policy provides coverage for a specific period of time, such as 10, 20, or 30 years. It is typically more affordable and offers a death benefit if the insured passes away during the term. A whole life insurance policy, on the other hand, provides coverage for the entire lifetime of the insured. It also includes a cash value component that grows over time and can be borrowed against or withdrawn. Whole life insurance premiums are usually higher than term life premiums.\n"
     ]
    }
   ],
   "source": [
    "print(conversation.memory.buffer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aW0TPWRBJ2w2"
   },
   "source": [
    "## Overloading the Memory\n",
    "\n",
    "When the conversation memory becomes full, the chatbot will begin to forget. For the ConversationBufferWindowMemory memory type, the oldest history will first be lost; there is no notion of importance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 820
    },
    "id": "d-a2iUpTJIYn",
    "outputId": "689496ee-16e2-44c4-fef4-30078d269b79"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Okay, then let me introduce myself, my name is Jeff\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Nice to meet you, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You have ONE JOB! Remember that my favorite color is blue.\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I apologize for the oversight. Thank you for reminding me that your favorite color is blue, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Do you remember my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Yes, Jeff! How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Do you remember my favorite color?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Yes, Jeff! Your favorite color is blue. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #0\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Thank you for letting me know about fact #0. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #1\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Thank you for letting me know about fact #1. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #2\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Thank you for letting me know about fact #2. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #3\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Thank you for letting me know about fact #3. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #4\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Thank you for letting me know about fact #4. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #5\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       " Thank you for letting me know about fact #5. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #6\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       " Thank you for letting me know about fact #6. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #7\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       " Thank you for letting me know about fact #7. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #8\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       " Thank you for letting me know about fact #8. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #9\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       " Thank you for letting me know about fact #9. How can I assist you today, Jeff?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Do you remember my name?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Yes, Jeff. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: Do you remember my favorite color?\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I'm sorry, Jeff. I don't have that information. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: OMG, you had one job!\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "I apologize for not remembering your favorite color, Jeff. How can I assist you today?"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "conversation = begin_conversation()\n",
    "chat(conversation, \"Okay, then let me introduce myself, my name is Jeff\")\n",
    "chat(conversation, \"You have ONE JOB! Remember that my favorite color is blue.\")\n",
    "chat(conversation, \"Do you remember my name?\")\n",
    "chat(conversation, \"Do you remember my favorite color?\")\n",
    "for i in range(10):\n",
    "  chat(conversation, f\"You need to remember fact #{i}\")\n",
    "chat(conversation, \"Do you remember my name?\")\n",
    "chat(conversation, \"Do you remember my favorite color?\")\n",
    "chat(conversation, \"OMG, you had one job!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gbi_-ywvX9NX"
   },
   "source": [
    "We can quickly look inside the memory of the LangChain-managed chat memory and see our conversation memory with the LLM. It becomes evident why it forgot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7YCKTQkhJlJB",
    "outputId": "7b4afb86-da75-4ae0-8e6e-e28510d86209"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: You need to remember fact #8\n",
      "AI:  Thank you for letting me know about fact #8. How can I assist you today, Jeff?\n",
      "Human: You need to remember fact #9\n",
      "AI:  Thank you for letting me know about fact #9. How can I assist you today, Jeff?\n",
      "Human: Do you remember my name?\n",
      "AI: Yes, Jeff. How can I assist you today?\n",
      "Human: Do you remember my favorite color?\n",
      "AI: I'm sorry, Jeff. I don't have that information. How can I assist you today?\n",
      "Human: OMG, you had one job!\n",
      "AI: I apologize for not remembering your favorite color, Jeff. How can I assist you today?\n"
     ]
    }
   ],
   "source": [
    "print(conversation.memory.buffer)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (genai)",
   "language": "python",
   "name": "genai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}