docs/gpt-4-langchain-docs.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "GFLLl1Agum8O"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pinecone-io/examples/blob/master/docs/gpt-4-langchain-docs.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/pinecone-io/examples/blob/master/docs/gpt-4-langchain-docs.ipynb)\n",
        "\n",
        "# GPT4 with Retrieval Augmentation over LangChain Docs\n",
        "\n",
        "In this notebook we'll work through an example of using GPT-4 with retrieval augmentation to answer questions about the LangChain Python library.\n",
        "\n",
        "[![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/full-link.svg)](https://github.com/pinecone-io/examples/blob/master/learn/generation/openai/gpt-4-langchain-docs.ipynb)\n",
        "\n",
        "To begin we must install the prerequisite libraries:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_HDKlQO5svqI",
        "outputId": "275263b8-6a09-4a7d-914c-c069b28784c4"
      },
      "outputs": [],
      "source": [
        "!pip install -qU \\\n",
        "    openai==1.66.3 \\\n",
        "    pinecone==5.4.2 \\\n",
        "    pinecone-datasets==1.0.2 \\\n",
        "    pinecone-notebooks==0.1.1 \\\n",
        "    tqdm"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bZdWl7kR9KWJ"
      },
      "source": [
        "---\n",
        "\n",
        "\ud83d\udea8 _Note: the above `pip install` is formatted for Jupyter notebooks. If running elsewhere you may need to drop the `!`._\n",
        "\n",
        "---"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "NgUEJ6vDum8q"
      },
      "source": [
        "In this example, we will use a pre-embedding dataset of the LangChain docs from [python.langchain.readthedocs.com/](https://python.langchain.com/en/latest/). If you'd like to see how we perform the data preparation refer to [this notebook]().\n",
        "\n",
        "The embeddings were produced with OpenAI's `text-embedding-ada-002` model which outputs embeddings with dimension `1536`.\n",
        "\n",
        "Let's go ahead and download the dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 393
        },
        "id": "phk1tQ0U92DM",
        "outputId": "2fa9baec-f4eb-4e32-c28a-34c4fffd146f"
      },
      "outputs": [],
      "source": [
        "from pinecone_datasets import load_dataset\n",
        "\n",
        "dataset = load_dataset('langchain-python-docs-text-embedding-ada-002')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Loading documents parquet files: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 2/2 [00:32<00:00, 16.08s/it]\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>values</th>\n",
              "      <th>metadata</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>417ede5d-39be-498f-b518-f47ed4e53b90</td>\n",
              "      <td>[0.005949743557721376, 0.01983247883617878, -0...</td>\n",
              "      <td>{'chunk': 0, 'text': '.rst\n",
              ".pdf\n",
              "Welcome to Lan...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>110f550d-110b-4378-b95e-141397fa21bc</td>\n",
              "      <td>[0.009401749819517136, 0.02443608082830906, 0....</td>\n",
              "      <td>{'chunk': 1, 'text': 'Use Cases#\n",
              "Best practice...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>d5f00f02-3295-4567-b297-5e3262dc2728</td>\n",
              "      <td>[-0.005517194513231516, 0.0208403542637825, 0....</td>\n",
              "      <td>{'chunk': 2, 'text': 'Gallery: A collection of...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0b6fe3c6-1f0e-4608-a950-43231e46b08a</td>\n",
              "      <td>[-0.006499645300209522, 0.0011573900701478124,...</td>\n",
              "      <td>{'chunk': 0, 'text': 'Search\n",
              "Error\n",
              "Please acti...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>39d5f15f-b973-42c0-8c9b-a2df49b627dc</td>\n",
              "      <td>[-0.005658374633640051, 0.00817849114537239, 0...</td>\n",
              "      <td>{'chunk': 0, 'text': '.md\n",
              ".pdf\n",
              "Dependents\n",
              "Depe...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                     id  \\\n",
              "0  417ede5d-39be-498f-b518-f47ed4e53b90   \n",
              "1  110f550d-110b-4378-b95e-141397fa21bc   \n",
              "2  d5f00f02-3295-4567-b297-5e3262dc2728   \n",
              "3  0b6fe3c6-1f0e-4608-a950-43231e46b08a   \n",
              "4  39d5f15f-b973-42c0-8c9b-a2df49b627dc   \n",
              "\n",
              "                                              values  \\\n",
              "0  [0.005949743557721376, 0.01983247883617878, -0...   \n",
              "1  [0.009401749819517136, 0.02443608082830906, 0....   \n",
              "2  [-0.005517194513231516, 0.0208403542637825, 0....   \n",
              "3  [-0.006499645300209522, 0.0011573900701478124,...   \n",
              "4  [-0.005658374633640051, 0.00817849114537239, 0...   \n",
              "\n",
              "                                            metadata  \n",
              "0  {'chunk': 0, 'text': '.rst\n",
              ".pdf\n",
              "Welcome to Lan...  \n",
              "1  {'chunk': 1, 'text': 'Use Cases#\n",
              "Best practice...  \n",
              "2  {'chunk': 2, 'text': 'Gallery: A collection of...  \n",
              "3  {'chunk': 0, 'text': 'Search\n",
              "Error\n",
              "Please acti...  \n",
              "4  {'chunk': 0, 'text': '.md\n",
              ".pdf\n",
              "Dependents\n",
              "Depe...  "
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# We drop the sparse_values column since it is not needed in this demo\n",
        "dataset.documents.drop(['sparse_values'], axis=1, inplace=True)\n",
        "\n",
        "# We rename the blob column to metadata\n",
        "dataset.documents.drop(['metadata'], axis=1, inplace=True)\n",
        "dataset.documents.rename(columns={'blob': 'metadata'}, inplace=True)\n",
        "\n",
        "dataset.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's take a look at what sort of metadata we're working with in this dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Here are some example entries in our Knowledge Base:\n",
            "\n",
            "{'chunk': 0,\n",
            " 'text': '.rst\\n'\n",
            "         '.pdf\\n'\n",
            "         'Welcome to LangChain\\n'\n",
            "         ' Contents \\n'\n",
            "         'Getting Started\\n'\n",
            "         'Modules\\n'\n",
            "         'Use Cases\\n'\n",
            "         'Reference Docs\\n'\n",
            "         'Ecosystem\\n'\n",
            "         'Additional Resources\\n'\n",
            "         'Welcome to LangChain#\\n'\n",
            "         'LangChain is a framework for developing applications powered by '\n",
            "         'language models. We believe that the most powerful and '\n",
            "         'differentiated applications will not only call out to a language '\n",
            "         'model, but will also be:\\n'\n",
            "         'Data-aware: connect a language model to other sources of data\\n'\n",
            "         'Agentic: allow a language model to interact with its environment\\n'\n",
            "         'The LangChain framework is designed around these principles.\\n'\n",
            "         'This is the Python specific portion of the documentation. For a '\n",
            "         'purely conceptual guide to LangChain, see here. For the JavaScript '\n",
            "         'documentation, see here.\\n'\n",
            "         'Getting Started#\\n'\n",
            "         'How to get started using LangChain to create an Language Model '\n",
            "         'application.\\n'\n",
            "         'Quickstart Guide\\n'\n",
            "         'Concepts and terminology.\\n'\n",
            "         'Concepts and terminology\\n'\n",
            "         'Tutorials created by community experts and presented on YouTube.\\n'\n",
            "         'Tutorials\\n'\n",
            "         'Modules#\\n'\n",
            "         'These modules are the core abstractions which we view as the '\n",
            "         'building blocks of any LLM-powered application.\\n'\n",
            "         'For each module LangChain provides standard, extendable interfaces. '\n",
            "         'LangChain also provides external integrations and even end-to-end '\n",
            "         'implementations for off-the-shelf use.\\n'\n",
            "         'The docs for each module contain quickstart examples, how-to guides, '\n",
            "         'reference docs, and conceptual guides.\\n'\n",
            "         'The modules are (from least to most complex):\\n'\n",
            "         'Models: Supported model types and integrations.\\n'\n",
            "         'Prompts: Prompt management, optimization, and serialization.\\n'\n",
            "         'Memory: Memory refers to state that is persisted between calls of a '\n",
            "         'chain/agent.\\n'\n",
            "         'Indexes: Language models become much more powerful when combined '\n",
            "         'with application-specific data - this module contains interfaces and '\n",
            "         'integrations for loading, querying and updating external data.\\n'\n",
            "         'Chains: Chains are structured sequences of calls (to an LLM or to a '\n",
            "         'different utility).\\n'\n",
            "         'Agents: An agent is a Chain in which an LLM, given a high-level '\n",
            "         'directive and a set of tools, repeatedly decides an action, executes '\n",
            "         'the action and observes the outcome until the high-level directive '\n",
            "         'is complete.\\n'\n",
            "         'Callbacks: Callbacks let you log and stream the intermediate steps '\n",
            "         'of any chain, making it easy to observe, debug, and evaluate the '\n",
            "         'internals of an application.\\n'\n",
            "         'Use Cases#\\n'\n",
            "         'Best practices and built-in implementations for common LangChain use '\n",
            "         'cases:',\n",
            " 'url': 'https://python.langchain.com/en/latest/index.html'}\n",
            "{'chunk': 1,\n",
            " 'text': 'Use Cases#\\n'\n",
            "         'Best practices and built-in implementations for common LangChain use '\n",
            "         'cases:\\n'\n",
            "         'Autonomous Agents: Autonomous agents are long-running agents that '\n",
            "         'take many steps in an attempt to accomplish an objective. Examples '\n",
            "         'include AutoGPT and BabyAGI.\\n'\n",
            "         'Agent Simulations: Putting agents in a sandbox and observing how '\n",
            "         'they interact with each other and react to events can be an '\n",
            "         'effective way to evaluate their long-range reasoning and planning '\n",
            "         'abilities.\\n'\n",
            "         'Personal Assistants: One of the primary LangChain use cases. '\n",
            "         'Personal assistants need to take actions, remember interactions, and '\n",
            "         'have knowledge about your data.\\n'\n",
            "         'Question Answering: Another common LangChain use case. Answering '\n",
            "         'questions over specific documents, only utilizing the information in '\n",
            "         'those documents to construct an answer.\\n'\n",
            "         'Chatbots: Language models love to chat, making this a very natural '\n",
            "         'use of them.\\n'\n",
            "         'Querying Tabular Data: Recommended reading if you want to use '\n",
            "         'language models to query structured data (CSVs, SQL, dataframes, '\n",
            "         'etc).\\n'\n",
            "         'Code Understanding: Recommended reading if you want to use language '\n",
            "         'models to analyze code.\\n'\n",
            "         'Interacting with APIs: Enabling language models to interact with '\n",
            "         'APIs is extremely powerful. It gives them access to up-to-date '\n",
            "         'information and allows them to take actions.\\n'\n",
            "         'Extraction: Extract structured information from text.\\n'\n",
            "         'Summarization: Compressing longer documents. A type of '\n",
            "         'Data-Augmented Generation.\\n'\n",
            "         'Evaluation: Generative models are hard to evaluate with traditional '\n",
            "         'metrics. One promising approach is to use language models themselves '\n",
            "         'to do the evaluation.\\n'\n",
            "         'Reference Docs#\\n'\n",
            "         'Full documentation on all methods, classes, installation methods, '\n",
            "         'and integration setups for LangChain.\\n'\n",
            "         'LangChain Installation\\n'\n",
            "         'Reference Documentation\\n'\n",
            "         'Ecosystem#\\n'\n",
            "         'LangChain integrates a lot of different LLMs, systems, and '\n",
            "         'products.\\n'\n",
            "         'From the other side, many systems and products depend on LangChain.\\n'\n",
            "         'It creates a vibrant and thriving ecosystem.\\n'\n",
            "         'Integrations: Guides for how other products can be used with '\n",
            "         'LangChain.\\n'\n",
            "         'Dependents: List of repositories that use LangChain.\\n'\n",
            "         'Deployments: A collection of instructions, code snippets, and '\n",
            "         'template repositories for deploying LangChain apps.\\n'\n",
            "         'Additional Resources#\\n'\n",
            "         'Additional resources we think may be useful as you develop your '\n",
            "         'application!\\n'\n",
            "         'LangChainHub: The LangChainHub is a place to share and explore other '\n",
            "         'prompts, chains, and agents.',\n",
            " 'url': 'https://python.langchain.com/en/latest/index.html'}\n",
            "{'chunk': 2,\n",
            " 'text': 'Gallery: A collection of great projects that use Langchain, compiled '\n",
            "         'by the folks at Kyrolabs. Useful for finding inspiration and example '\n",
            "         'implementations.\\n'\n",
            "         'Tracing: A guide on using tracing in LangChain to visualize the '\n",
            "         'execution of chains and agents.\\n'\n",
            "         'Model Laboratory: Experimenting with different prompts, models, and '\n",
            "         'chains is a big part of developing the best possible application. '\n",
            "         'The ModelLaboratory makes it easy to do so.\\n'\n",
            "         'Discord: Join us on our Discord to discuss all things LangChain!\\n'\n",
            "         'YouTube: A collection of the LangChain tutorials and videos.\\n'\n",
            "         'Production Support: As you move your LangChains into production, '\n",
            "         'we\u2019d love to offer more comprehensive support. Please fill out this '\n",
            "         'form and we\u2019ll set up a dedicated support Slack channel.\\n'\n",
            "         'next\\n'\n",
            "         'Quickstart Guide\\n'\n",
            "         ' Contents\\n'\n",
            "         '  \\n'\n",
            "         'Getting Started\\n'\n",
            "         'Modules\\n'\n",
            "         'Use Cases\\n'\n",
            "         'Reference Docs\\n'\n",
            "         'Ecosystem\\n'\n",
            "         'Additional Resources\\n'\n",
            "         'By Harrison Chase\\n'\n",
            "         '    \\n'\n",
            "         '      \u00a9 Copyright 2023, Harrison Chase.\\n'\n",
            "         '      \\n'\n",
            "         '  Last updated on May 30, 2023.',\n",
            " 'url': 'https://python.langchain.com/en/latest/index.html'}\n"
          ]
        }
      ],
      "source": [
        "from pprint import pprint\n",
        "\n",
        "print(\"Here are some example entries in our Knowledge Base:\\n\")\n",
        "for r in dataset.documents.iloc[0:3].to_dict(orient=\"records\"):\n",
        "    pprint(r['metadata'])"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "JegURaAg2PuN"
      },
      "source": [
        "Our chunks are ready so now we move onto embedding and indexing everything."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initializing the Pinecone client\n",
        "\n",
        "Now the data is ready, we can set up our index to store it.\n",
        "\n",
        "We begin by instantiating the Pinecone client. To do this we need a [free API key](https://app.pinecone.io)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "if not os.environ.get(\"PINECONE_API_KEY\"):\n",
        "    from pinecone_notebooks.colab import Authenticate\n",
        "    Authenticate()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {},
      "outputs": [],
      "source": [
        "from pinecone import Pinecone\n",
        "\n",
        "api_key = os.environ.get(\"PINECONE_API_KEY\")\n",
        "\n",
        "# Configure client\n",
        "pc = Pinecone(api_key=api_key)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Creating a Pinecone Index\n",
        "\n",
        "When creating the index we need to define several configuration properties. \n",
        "\n",
        "- `name` can be anything we like. The name is used as an identifier for the index when performing other operations such as `describe_index`, `delete_index`, and so on. \n",
        "- `metric` specifies the similarity metric that will be used later when you make queries to the index.\n",
        "- `dimension` should correspond to the dimension of the dense vectors produced by your embedding model. In this quick start, we are using made-up data so a small value is simplest.\n",
        "- `spec` holds a specification which tells Pinecone how you would like to deploy our index. You can find a list of all [available providers and regions here](https://docs.pinecone.io/docs/projects).\n",
        "\n",
        "There are more configurations available, but this minimal set will get us started."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EO8sbJFZNyIZ",
        "outputId": "864ac7a6-25fe-455b-9b8a-8b45f583987c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{\n",
              "    \"name\": \"gpt-4-langchain-docs-fast\",\n",
              "    \"dimension\": 1536,\n",
              "    \"metric\": \"cosine\",\n",
              "    \"host\": \"gpt-4-langchain-docs-fast-dojoi3u.svc.aped-4627-b74a.pinecone.io\",\n",
              "    \"spec\": {\n",
              "        \"serverless\": {\n",
              "            \"cloud\": \"aws\",\n",
              "            \"region\": \"us-east-1\"\n",
              "        }\n",
              "    },\n",
              "    \"status\": {\n",
              "        \"ready\": true,\n",
              "        \"state\": \"Ready\"\n",
              "    },\n",
              "    \"deletion_protection\": \"disabled\"\n",
              "}"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from pinecone import ServerlessSpec\n",
        "\n",
        "index_name = 'gpt-4-langchain-docs-fast'\n",
        "\n",
        "# check if index already exists (it shouldn't if this is first time)\n",
        "if not pc.has_index(name=index_name):\n",
        "    # if does not exist, create index\n",
        "    pc.create_index(\n",
        "        name=index_name,\n",
        "        dimension=1536,  # dimensionality of text-embedding-ada-002\n",
        "        metric='cosine',\n",
        "        spec=ServerlessSpec(cloud='aws', region='us-east-1')\n",
        "    )\n",
        "\n",
        "pc.describe_index(name=index_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Storing data in the Index\n",
        "\n",
        "First we need to instantiate an Index client that can interact with the index we just created."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'dimension': 1536,\n",
              " 'index_fullness': 0.0,\n",
              " 'namespaces': {},\n",
              " 'total_vector_count': 0}"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Instantiate an Index client\n",
        "index = pc.Index(name=index_name)\n",
        "\n",
        "# View index stats for the new index\n",
        "index.describe_index_stats()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ezSTzN2rPa2o"
      },
      "source": [
        "We can see the index is currently empty with a `total_vector_count` of `0`. We can begin populating it with OpenAI `text-embedding-ada-002` built embeddings like so:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 155,
          "referenced_widgets": [
            "9b9b464993e24f2ba80c85d30f1f2ae4",
            "38632470c650411e937b7b3893450ac4",
            "65a3ad0c971c4a848e742263313af53b",
            "9a03775c460d4931bd911b4bcdde5b50",
            "0499e771fc7c4c44aaa34715b695db14",
            "683885a9181244098883b5e4180ec447",
            "83f37372965746faa21cf43dad69fede",
            "c8d51d1b53714647b960bceb42e16867",
            "0a447a43050642748e8bef0af11dd8dc",
            "5510616e20374a92aa76037557203def",
            "9ea3f496115f4663b4596efeafcc18f6",
            "b453cc511c7747398718ab8ec11cdaca",
            "407bb220f41c47429352c19922459858",
            "3709f6f136914c5f90cd21bbb6fc0dda",
            "e7138b87ce8c49d881b878f6ca333c3d",
            "3f1d7e92a02b40b58ccefb7c8ce7f301",
            "b415838f711645a688afa5628850bfe0",
            "ab8c9025f140460a999792ffe027ea86",
            "4ad1a29b9de8478eb45dc7a050cce5f7",
            "fb751aefaa9641c186a698bf961c9a20",
            "fb0384555d1040a990bb71d498050ff1",
            "eeb5ba13750d432bb1424349abb9fc7d"
          ]
        },
        "id": "iZbFbulAPeop",
        "outputId": "087596ce-d125-4875-8cc5-7931caa085c5"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "sending upsert requests: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 6952/6952 [01:00<00:00, 114.03it/s]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "{'upserted_count': 6952}"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "index.upsert_from_dataframe(\n",
        "    df=dataset.documents, \n",
        "    batch_size=100\n",
        ")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "YttJOrEtQIF9"
      },
      "source": [
        "Now we've added all of our langchain docs to the index. With that we can move on to retrieval and then answer generation using GPT-4."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "FumVmMRlQQ7w"
      },
      "source": [
        "## Retrieval"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "nLRODeL-QTJ9"
      },
      "source": [
        "To search through our documents we first need to create a query vector `xq`. Using `xq` we will retrieve the most relevant chunks from the LangChain docs. To create that query vector we must initialize a `text-embedding-ada-002` embedding model with OpenAI. For this, you need an [OpenAI API key](https://platform.openai.com/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "dhDTQnqp-yyx"
      },
      "outputs": [],
      "source": [
        "def create_embedding(query):\n",
        "    from openai import OpenAI\n",
        "\n",
        "    # Get OpenAI api key from platform.openai.com\n",
        "    openai_api_key = os.getenv('OPENAI_API_KEY') or 'sk-...'\n",
        "\n",
        "    # Instantiate the OpenAI client\n",
        "    client = OpenAI(api_key=openai_api_key)\n",
        "\n",
        "    # Create an embedding\n",
        "    res = client.embeddings.create(\n",
        "      model=\"text-embedding-ada-002\",\n",
        "      input=[query],\n",
        "    )\n",
        "    return res.data[0].embedding"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "FMUPdX9cQQYC"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'matches': [{'id': '2f66a6a5-c829-4118-acb8-f08667f3f95d',\n",
              "              'metadata': {'chunk': 2.0,\n",
              "                           'text': 'for full documentation on:\\\\n\\\\nGetting '\n",
              "                                   'started (installation, setting up the '\n",
              "                                   'environment, simple examples)\\\\n\\\\nHow-To '\n",
              "                                   'examples (demos, integrations, helper '\n",
              "                                   'functions)\\\\n\\\\nReference (full API '\n",
              "                                   'docs)\\\\n\\\\nResources (high-level '\n",
              "                                   'explanation of core '\n",
              "                                   'concepts)\\\\n\\\\n\u00f0\\\\x9f\\\\x9a\\\\x80 What can '\n",
              "                                   'this help with?\\\\n\\\\nThere are six main '\n",
              "                                   'areas that LangChain is designed to help '\n",
              "                                   'with.\\\\nThese are, in increasing order of '\n",
              "                                   'complexity:\\\\n\\\\n\u00f0\\\\x9f\u201c\\\\x83 LLMs and '\n",
              "                                   'Prompts:\\\\n\\\\nThis includes prompt '\n",
              "                                   'management, prompt optimization, a generic '\n",
              "                                   'interface for all LLMs, and common '\n",
              "                                   'utilities for working with '\n",
              "                                   'LLMs.\\\\n\\\\n\u00f0\\\\x9f\u201d\\\\x97 '\n",
              "                                   'Chains:\\\\n\\\\nChains go beyond a single LLM '\n",
              "                                   'call and involve sequences of calls '\n",
              "                                   '(whether to an LLM or a different '\n",
              "                                   'utility). LangChain provides a standard '\n",
              "                                   'interface for chains, lots of integrations '\n",
              "                                   'with other tools, and end-to-end chains '\n",
              "                                   'for common applications.\\\\n\\\\n\u00f0\\\\x9f\u201c\\\\x9a '\n",
              "                                   'Data Augmented Generation:\\\\n\\\\nData '\n",
              "                                   'Augmented Generation involves specific '\n",
              "                                   'types of chains that first interact with '\n",
              "                                   'an external data source to fetch data for '\n",
              "                                   'use in the generation step. Examples '\n",
              "                                   'include summarization of long pieces of '\n",
              "                                   'text and question/answering over specific '\n",
              "                                   'data sources.\\\\n\\\\n\u00f0\\\\x9f\u00a4\\\\x96 '\n",
              "                                   'Agents:\\\\n\\\\nAgents involve an LLM making '\n",
              "                                   'decisions about which Actions to take, '\n",
              "                                   'taking that Action, seeing an Observation, '\n",
              "                                   'and repeating that until done. LangChain',\n",
              "                           'url': 'https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/markdown.html'},\n",
              "              'score': 0.869625509,\n",
              "              'values': []},\n",
              "             {'id': 'e26407fd-df5d-4e88-a59e-2ffd2b2b87d6',\n",
              "              'metadata': {'chunk': 17.0,\n",
              "                           'text': 'an Observation, and repeating that until '\n",
              "                                   'done. LangChain provides a standard '\n",
              "                                   'interface for agents, a selection of '\n",
              "                                   'agents to choose from, and examples of end '\n",
              "                                   'to end agents.\\\\n\\\\n\\\\n\\\\n\\\\n\\\\nUse '\n",
              "                                   'Cases#\\\\nThe above modules can be used in '\n",
              "                                   'a variety of ways. LangChain also provides '\n",
              "                                   'guidance and assistance in this. Below are '\n",
              "                                   'some of the common use cases LangChain '\n",
              "                                   'supports.\\\\n\\\\nPersonal Assistants: The '\n",
              "                                   'main LangChain use case. Personal '\n",
              "                                   'assistants need to take actions, remember '\n",
              "                                   'interactions, and have knowledge about '\n",
              "                                   'your data.\\\\nQuestion Answering: The '\n",
              "                                   'second big LangChain use case. Answering '\n",
              "                                   'questions over specific documents, only '\n",
              "                                   'utilizing the information in those '\n",
              "                                   'documents to construct an '\n",
              "                                   'answer.\\\\nChatbots: Since language models '\n",
              "                                   'are good at producing text, that makes '\n",
              "                                   'them ideal for creating '\n",
              "                                   'chatbots.\\\\nQuerying Tabular Data: If you '\n",
              "                                   'want to understand how to use LLMs to '\n",
              "                                   'query data that is stored in a tabular '\n",
              "                                   'format (csvs, SQL, dataframes, etc) you '\n",
              "                                   'should read this page.\\\\nInteracting with '\n",
              "                                   'APIs: Enabling LLMs to interact with APIs '\n",
              "                                   'is extremely powerful in order to give '\n",
              "                                   'them more up-to-date information and allow '\n",
              "                                   'them to take actions.\\\\nExtraction: '\n",
              "                                   'Extract structured information from '\n",
              "                                   'text.\\\\nSummarization: Summarizing longer '\n",
              "                                   'documents into shorter, more condensed '\n",
              "                                   'chunks of information. A type of Data '\n",
              "                                   'Augmented Generation.\\\\nEvaluation: '\n",
              "                                   'Generative models are notoriously',\n",
              "                           'url': 'https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/sitemap.html'},\n",
              "              'score': 0.857345343,\n",
              "              'values': []},\n",
              "             {'id': 'd578f3d3-8107-4fb1-87cd-0c81923761f7',\n",
              "              'metadata': {'chunk': 7.0,\n",
              "                           'text': 'working with raw text, they work with '\n",
              "                                   'messages. LangChain provides a standard '\n",
              "                                   'interface for working with them and doing '\n",
              "                                   'all the same things as '\n",
              "                                   'above.\\\\n\\\\n\\\\n\\\\n\\\\n\\\\nUse Cases#\\\\nThe '\n",
              "                                   'above modules can be used in a variety of '\n",
              "                                   'ways. LangChain also provides guidance and '\n",
              "                                   'assistance in this. Below are some of the '\n",
              "                                   'common use cases LangChain '\n",
              "                                   'supports.\\\\n\\\\nAgents: Agents are systems '\n",
              "                                   'that use a language model to interact with '\n",
              "                                   'other tools. These can be used to do more '\n",
              "                                   'grounded question/answering, interact with '\n",
              "                                   'APIs, or even take actions.\\\\nChatbots: '\n",
              "                                   'Since language models are good at '\n",
              "                                   'producing text, that makes them ideal for '\n",
              "                                   'creating chatbots.\\\\nData Augmented '\n",
              "                                   'Generation: Data Augmented Generation '\n",
              "                                   'involves specific types of chains that '\n",
              "                                   'first interact with an external datasource '\n",
              "                                   'to fetch data to use in the generation '\n",
              "                                   'step. Examples of this include '\n",
              "                                   'summarization of long pieces of text and '\n",
              "                                   'question/answering over specific data '\n",
              "                                   'sources.\\\\nQuestion Answering: Answering '\n",
              "                                   'questions over specific documents, only '\n",
              "                                   'utilizing the information in those '\n",
              "                                   'documents to construct an answer. A type '\n",
              "                                   'of Data Augmented '\n",
              "                                   'Generation.\\\\nSummarization: Summarizing '\n",
              "                                   'longer documents into shorter, more '\n",
              "                                   'condensed chunks of information. A type of '\n",
              "                                   'Data Augmented Generation.\\\\nQuerying '\n",
              "                                   'Tabular Data: If you want to understand '\n",
              "                                   'how to use LLMs to query data that is '\n",
              "                                   'stored in a tabular format (csvs,',\n",
              "                           'url': 'https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/sitemap.html'},\n",
              "              'score': 0.85538429,\n",
              "              'values': []},\n",
              "             {'id': '2a19b051-2bf3-4675-ac84-8cfb859c9fc7',\n",
              "              'metadata': {'chunk': 1.0,\n",
              "                           'text': 'Initiate the LLMChain\\n'\n",
              "                                   'Run the LLMChain\\n'\n",
              "                                   'By Harrison Chase\\n'\n",
              "                                   '    \\n'\n",
              "                                   '      \u00a9 Copyright 2023, Harrison Chase.\\n'\n",
              "                                   '      \\n'\n",
              "                                   '  Last updated on May 30, 2023.',\n",
              "                           'url': 'https://python.langchain.com/en/latest/modules/models/llms/integrations/pipelineai_example.html'},\n",
              "              'score': 0.85139519,\n",
              "              'values': []},\n",
              "             {'id': 'badbba15-bc60-4177-89c0-f7f743a7c655',\n",
              "              'metadata': {'chunk': 2.0,\n",
              "                           'text': 'use memory.\\\\nIndexes: Language models are '\n",
              "                                   'often more powerful when combined with '\n",
              "                                   'your own text data - this module covers '\n",
              "                                   'best practices for doing exactly '\n",
              "                                   'that.\\\\nChains: Chains go beyond just a '\n",
              "                                   'single LLM call, and are sequences of '\n",
              "                                   'calls (whether to an LLM or a different '\n",
              "                                   'utility). LangChain provides a standard '\n",
              "                                   'interface for chains, lots of integrations '\n",
              "                                   'with other tools, and end-to-end chains '\n",
              "                                   'for common applications.\\\\nAgents: Agents '\n",
              "                                   'involve an LLM making decisions about '\n",
              "                                   'which Actions to take, taking that Action, '\n",
              "                                   'seeing an Observation, and repeating that '\n",
              "                                   'until done. LangChain provides a standard '\n",
              "                                   'interface for agents, a selection of '\n",
              "                                   'agents to choose from, and examples of end '\n",
              "                                   'to end agents.\\\\nUse Cases\\\\nThe above '\n",
              "                                   'modules can be used in a variety of ways. '\n",
              "                                   'LangChain also provides guidance and '\n",
              "                                   'assistance in this. Below are some of the '\n",
              "                                   'common use cases LangChain '\n",
              "                                   'supports.\\\\nPersonal Assistants: The main '\n",
              "                                   'LangChain use case. Personal assistants '\n",
              "                                   'need to take actions, remember '\n",
              "                                   'interactions, and have knowledge about '\n",
              "                                   'your data.\\\\nQuestion Answering: The '\n",
              "                                   'second big LangChain use case. Answering '\n",
              "                                   'questions over specific documents, only '\n",
              "                                   'utilizing the information in those '\n",
              "                                   'documents to construct an '\n",
              "                                   'answer.\\\\nChatbots: Since language models '\n",
              "                                   'are good at producing text, that makes '\n",
              "                                   'them ideal for creating '\n",
              "                                   'chatbots.\\\\nQuerying Tabular Data: If you '\n",
              "                                   'want to understand how to',\n",
              "                           'url': 'https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/diffbot.html'},\n",
              "              'score': 0.84972322,\n",
              "              'values': []}],\n",
              " 'namespace': '',\n",
              " 'usage': {'read_units': 6}}"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "query = \"how do I use the LLMChain in LangChain?\"\n",
        "\n",
        "# retrieve from Pinecone\n",
        "xq = create_embedding(query)\n",
        "\n",
        "# get relevant contexts (including the questions)\n",
        "res = index.query(vector=xq, top_k=5, include_metadata=True)\n",
        "res"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "MoBSiDLIUADZ"
      },
      "source": [
        "With retrieval complete, we move on to feeding these into GPT-4 to produce answers."
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "qfzS4-6-UXgX"
      },
      "source": [
        "## Retrieval Augmented Generation"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "XPC1jQaKUcy0"
      },
      "source": [
        "GPT-4 is currently accessed via the `ChatCompletions` endpoint of OpenAI. \n",
        "\n",
        "To get a richer response from the LLM that includes context from our knowledge base, we need to retrieve context relevant to the query and then include it into the chat completion prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "unZstoHNUHeG"
      },
      "outputs": [],
      "source": [
        "def retrieval_augmented_prompt(query):\n",
        "    context_limit = 3750\n",
        "    xq = create_embedding(query)\n",
        "\n",
        "    # Get relevant contexts\n",
        "    query_results = index.query(vector=xq, top_k=3, include_metadata=True)\n",
        "    contexts = [\n",
        "        x.metadata['text'] for x in query_results.matches\n",
        "    ]\n",
        "\n",
        "    # Build our prompt with the retrieved contexts included\n",
        "    prompt_start = (\n",
        "        \"Answer the question based on the context below.\\n\\n\"+\n",
        "        \"Context:\\n\"\n",
        "    )\n",
        "    prompt_end = (\n",
        "        f\"\\n\\nQuestion: {query}\\nAnswer:\"\n",
        "    )\n",
        "    context_separator = \"\\n\\n---\\n\\n\"\n",
        "\n",
        "    # Join contexts and trim to fit within limit\n",
        "    combined_contexts = []\n",
        "    total_length = 0\n",
        "    \n",
        "    for context in contexts:\n",
        "        new_length = total_length + len(context) + len(context_separator)\n",
        "        if new_length >= context_limit:\n",
        "            break\n",
        "        combined_contexts.append(context)\n",
        "        total_length = new_length\n",
        "    \n",
        "    return prompt_start + context_separator.join(combined_contexts) + prompt_end"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LRcEHm0Z9fXE",
        "outputId": "fd3f082a-9c04-4ee6-8eda-8cb588810fad"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Answer the question based on the context below.\n",
            "\n",
            "Context:\n",
            "for full documentation on:\\n\\nGetting started (installation, setting up the environment, simple examples)\\n\\nHow-To examples (demos, integrations, helper functions)\\n\\nReference (full API docs)\\n\\nResources (high-level explanation of core concepts)\\n\\n\u00f0\\x9f\\x9a\\x80 What can this help with?\\n\\nThere are six main areas that LangChain is designed to help with.\\nThese are, in increasing order of complexity:\\n\\n\u00f0\\x9f\u201c\\x83 LLMs and Prompts:\\n\\nThis includes prompt management, prompt optimization, a generic interface for all LLMs, and common utilities for working with LLMs.\\n\\n\u00f0\\x9f\u201d\\x97 Chains:\\n\\nChains go beyond a single LLM call and involve sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.\\n\\n\u00f0\\x9f\u201c\\x9a Data Augmented Generation:\\n\\nData Augmented Generation involves specific types of chains that first interact with an external data source to fetch data for use in the generation step. Examples include summarization of long pieces of text and question/answering over specific data sources.\\n\\n\u00f0\\x9f\u00a4\\x96 Agents:\\n\\nAgents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain\n",
            "\n",
            "---\n",
            "\n",
            "an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.\\n\\n\\n\\n\\n\\nUse Cases#\\nThe above modules can be used in a variety of ways. LangChain also provides guidance and assistance in this. Below are some of the common use cases LangChain supports.\\n\\nPersonal Assistants: The main LangChain use case. Personal assistants need to take actions, remember interactions, and have knowledge about your data.\\nQuestion Answering: The second big LangChain use case. Answering questions over specific documents, only utilizing the information in those documents to construct an answer.\\nChatbots: Since language models are good at producing text, that makes them ideal for creating chatbots.\\nQuerying Tabular Data: If you want to understand how to use LLMs to query data that is stored in a tabular format (csvs, SQL, dataframes, etc) you should read this page.\\nInteracting with APIs: Enabling LLMs to interact with APIs is extremely powerful in order to give them more up-to-date information and allow them to take actions.\\nExtraction: Extract structured information from text.\\nSummarization: Summarizing longer documents into shorter, more condensed chunks of information. A type of Data Augmented Generation.\\nEvaluation: Generative models are notoriously\n",
            "\n",
            "Question: how do I use the LLMChain in LangChain?\n",
            "Answer:\n"
          ]
        }
      ],
      "source": [
        "prompt = retrieval_augmented_prompt(query)\n",
        "print(prompt)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "sihH_GMiV5_p"
      },
      "source": [
        "Now we ask the question of the LLM using chat completion:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "IThBqBi8V70d"
      },
      "outputs": [],
      "source": [
        "def chat_completion(prompt):\n",
        "    from openai import OpenAI\n",
        "\n",
        "    # Get OpenAI api key from platform.openai.com\n",
        "    openai_api_key = os.getenv('OPENAI_API_KEY') or 'sk-...'\n",
        "\n",
        "    # Instantiate the OpenAI client\n",
        "    client = OpenAI(api_key=openai_api_key)\n",
        "    \n",
        "    # Instructions\n",
        "    sys_prompt = f\"\"\"You are Q&A bot. A highly intelligent system that answers\n",
        "    user questions based on the information provided by the user above\n",
        "    each question. If the information can not be found in the information\n",
        "    provided by the user you truthfully say \"I don't know\".\n",
        "    \"\"\"\n",
        "    \n",
        "    res = client.chat.completions.create(\n",
        "        model='gpt-4o-mini-2024-07-18',\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": sys_prompt},\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ],\n",
        "        temperature=0\n",
        "    )\n",
        "    return res.choices[0].message.content.strip()\n",
        "\n",
        "def rag(query):\n",
        "    prompt = retrieval_augmented_prompt(query)\n",
        "    return chat_completion(prompt)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "\"To build a chatbot with LangChain, you can follow these general steps:\\n\\n1. **Set Up Your Environment**: Ensure you have Python or JavaScript installed, as LangChain supports these programming languages.\\n\\n2. **Install LangChain**: Use pip for Python or npm for JavaScript to install the LangChain framework.\\n\\n3. **Define Your Chatbot Logic**: Utilize LangChain's abstractions to define how your chatbot will process user inputs and generate responses. This may involve setting up a conversational flow and integrating with a language model like OpenAI's GPT-3.\\n\\n4. **Integrate Data Sources**: If your chatbot needs to access specific information, you can connect it to databases or APIs using LangChain's capabilities.\\n\\n5. **Test Your Chatbot**: Run your application and test the chatbot's responses to ensure it behaves as expected.\\n\\n6. **Deploy Your Chatbot**: Once satisfied with its performance, deploy your chatbot to a platform where users can interact with it.\\n\\nLangChain provides the necessary tools and frameworks to simplify these steps, making it easier to develop a chatbot powered by large language models.\""
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "answer = rag(\"How do I build a chatbot with LangChain?\")\n",
        "answer"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "QvS1yJhOWpiJ"
      },
      "source": [
        "To display this response nicely, we will display it in markdown."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "RDo2qeMHWto1",
        "outputId": "0c8157b4-9753-4f4a-a65c-db5c57a8b857"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "To build a chatbot with LangChain, you can follow these general steps:\n",
              "\n",
              "1. **Set Up Your Environment**: Ensure you have Python or JavaScript installed, as LangChain supports these programming languages.\n",
              "\n",
              "2. **Install LangChain**: Use pip for Python or npm for JavaScript to install the LangChain framework.\n",
              "\n",
              "3. **Define Your Chatbot Logic**: Utilize LangChain's abstractions to define how your chatbot will process user inputs and generate responses. This may involve setting up a conversational flow and integrating with a language model like OpenAI's GPT-3.\n",
              "\n",
              "4. **Integrate Data Sources**: If your chatbot needs to access specific information, you can connect it to databases or APIs using LangChain's capabilities.\n",
              "\n",
              "5. **Test Your Chatbot**: Run your application and test the chatbot's responses to ensure it behaves as expected.\n",
              "\n",
              "6. **Deploy Your Chatbot**: Once satisfied with its performance, deploy your chatbot to a platform where users can interact with it.\n",
              "\n",
              "LangChain provides the necessary tools and frameworks to simplify these steps, making it easier to develop a chatbot powered by large language models."
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from IPython.display import Markdown\n",
        "\n",
        "display(Markdown(answer))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "eJ-a8MHg0eYQ"
      },
      "source": [
        "Let's compare this to a non-augmented query..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 46
        },
        "id": "vwhaSgdF0ZDX",
        "outputId": "0ff1c0f3-2d77-4f32-b212-09463a4b9605"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "I don't know."
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def non_augmented_prompt(query):\n",
        "    return f\"\"\"\n",
        "Question: {query}\n",
        "Answer:\n",
        "\"\"\"\n",
        "\n",
        "answer2 = chat_completion(non_augmented_prompt(\"How do I create a chatbot with Langchain?\"))\n",
        "\n",
        "display(Markdown(answer2))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "5CSsA-dW0m_P"
      },
      "source": [
        "If we drop the `\"I don't know\"` part of the `sys_prompt`, the LLM will try to pull an answer out of things it already knows. These may or may not be correct."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 706
        },
        "id": "Z3svdTCZ0iJ2",
        "outputId": "40af56fe-a633-48bf-b732-cedf25ebea4b"
      },
      "outputs": [],
      "source": [
        "def hallucinating_chat_completion(prompt):\n",
        "    from openai import OpenAI\n",
        "\n",
        "    # Get OpenAI api key from platform.openai.com\n",
        "    openai_api_key = os.getenv('OPENAI_API_KEY') or 'sk-...'\n",
        "\n",
        "    # Instantiate the OpenAI client\n",
        "    client = OpenAI(api_key=openai_api_key)\n",
        "    \n",
        "    # Instructions\n",
        "    sys_prompt = f\"\"\"You are helpful Q&A bot.\"\"\"\n",
        "    \n",
        "    res = client.chat.completions.create(\n",
        "        model='gpt-4o-mini-2024-07-18',\n",
        "        messages=[\n",
        "            {\"role\": \"system\", \"content\": sys_prompt},\n",
        "            {\"role\": \"user\", \"content\": prompt}\n",
        "        ],\n",
        "        temperature=0\n",
        "    )\n",
        "    return res.choices[0].message.content.strip()\n",
        "\n",
        "answer3 = hallucinating_chat_completion(non_augmented_prompt(\"How do I create a chatbot with Langchain?\"))\n",
        "display(Markdown(answer3))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "GcGon5672lBb"
      },
      "source": [
        "Then we see something even worse than `\"I don't know\"` \u2014 hallucinations. Clearly augmenting our queries with additional context can make a huge difference to the performance of our system and ensure that trusted information is given priority when composing a response.\n",
        "\n",
        "Great, we've seen how to augment GPT-4 with semantic search to allow us to answer LangChain specific queries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Demo cleanup\n",
        "\n",
        "Once you're finished, we delete the index to save resources."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "Ah_vfEHV2khx"
      },
      "outputs": [],
      "source": [
        "pc.delete_index(name=index_name)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "iEUMlO8M2h4Y"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}