Add asyncio support for LLM (OpenAI), Chain (LLMChain, LLMMathChain), and Agent

docs/modules/agents/how_to_guides.rst

@@ -17,6 +17,7 @@ The first category of how-to guides here cover specific parts of working with ag
 
 `Max Iterations <./examples/max_iterations.html>`_: How to restrict an agent to a certain number of iterations.
 
+`Asynchronous <./examples/async_agent.html>`_: Covering asynchronous functionality.
 
 The next set of examples are all end-to-end agents for specific applications.
 In all examples there is an Agent with a particular set of tools.

docs/modules/chains/async_chain.ipynb

@@ -0,0 +1,132 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "593f7553-7038-498e-96d4-8255e5ce34f0",
+   "metadata": {},
+   "source": [
+    "# Async API for Chain\n",
+    "\n",
+    "LangChain provides async support for Chains by leveraging the [asyncio](https://docs.python.org/3/library/asyncio.html) library.\n",
+    "\n",
+    "Async methods are currently supported in `LLMChain` (through `arun`, `apredict`, `acall`) and `LLMMathChain` (through `arun` and `acall`). Async support for other chains is on the roadmap."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "c19c736e-ca74-4726-bb77-0a849bcc2960",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "BrightSmile Toothpaste Company\n",
+      "\n",
+      "\n",
+      "BrightSmile Toothpaste Co.\n",
+      "\n",
+      "\n",
+      "BrightSmile Toothpaste\n",
+      "\n",
+      "\n",
+      "Gleaming Smile Inc.\n",
+      "\n",
+      "\n",
+      "SparkleSmile Toothpaste\n",
+      "\u001b[1mConcurrent executed in 1.54 seconds.\u001b[0m\n",
+      "\n",
+      "\n",
+      "BrightSmile Toothpaste Co.\n",
+      "\n",
+      "\n",
+      "MintyFresh Toothpaste Co.\n",
+      "\n",
+      "\n",
+      "SparkleSmile Toothpaste.\n",
+      "\n",
+      "\n",
+      "Pearly Whites Toothpaste Co.\n",
+      "\n",
+      "\n",
+      "BrightSmile Toothpaste.\n",
+      "\u001b[1mSerial executed in 6.38 seconds.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "import asyncio\n",
+    "import time\n",
+    "\n",
+    "from langchain.llms import OpenAI\n",
+    "from langchain.prompts import PromptTemplate\n",
+    "from langchain.chains import LLMChain\n",
+    "\n",
+    "\n",
+    "def generate_serially():\n",
+    "    llm = OpenAI(temperature=0.9)\n",
+    "    prompt = PromptTemplate(\n",
+    "        input_variables=[\"product\"],\n",
+    "        template=\"What is a good name for a company that makes {product}?\",\n",
+    "    )\n",
+    "    chain = LLMChain(llm=llm, prompt=prompt)\n",
+    "    for _ in range(5):\n",
+    "        resp = chain.run(product=\"toothpaste\")\n",
+    "        print(resp)\n",
+    "\n",
+    "\n",
+    "async def async_generate(chain):\n",
+    "    resp = await chain.arun(product=\"toothpaste\")\n",
+    "    print(resp)\n",
+    "\n",
+    "\n",
+    "async def generate_concurrently():\n",
+    "    llm = OpenAI(temperature=0.9)\n",
+    "    prompt = PromptTemplate(\n",
+    "        input_variables=[\"product\"],\n",
+    "        template=\"What is a good name for a company that makes {product}?\",\n",
+    "    )\n",
+    "    chain = LLMChain(llm=llm, prompt=prompt)\n",
+    "    tasks = [async_generate(chain) for _ in range(5)]\n",
+    "    await asyncio.gather(*tasks)\n",
+    "\n",
+    "s = time.perf_counter()\n",
+    "# If running this outside of Jupyter, use asyncio.run(generate_concurrently())\n",
+    "await generate_concurrently()\n",
+    "elapsed = time.perf_counter() - s\n",
+    "print('\\033[1m' + f\"Concurrent executed in {elapsed:0.2f} seconds.\" + '\\033[0m')\n",
+    "\n",
+    "s = time.perf_counter()\n",
+    "generate_serially()\n",
+    "elapsed = time.perf_counter() - s\n",
+    "print('\\033[1m' + f\"Serial executed in {elapsed:0.2f} seconds.\" + '\\033[0m')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/chains/how_to_guides.rst

@@ -9,6 +9,7 @@ They are broken up into three categories:
 1. `Generic Chains <./generic_how_to.html>`_: Generic chains, that are meant to help build other chains rather than serve a particular purpose.
 2. `CombineDocuments Chains <./combine_docs_how_to.html>`_: Chains aimed at making it easy to work with documents (question answering, summarization, etc).
 3. `Utility Chains <./utility_how_to.html>`_: Chains consisting of an LLMChain interacting with a specific util.
+4. `Asynchronous <./async_chain.html>`_: Covering asynchronous functionality.
 
 .. toctree::
    :maxdepth: 1

docs/modules/llms/async_llm.ipynb

@@ -0,0 +1,150 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "f6574496-b360-4ffa-9523-7fd34a590164",
+   "metadata": {},
+   "source": [
+    "# Async API for LLM\n",
+    "\n",
+    "LangChain provides async support for LLMs by leveraging the [asyncio](https://docs.python.org/3/library/asyncio.html) library.\n",
+    "\n",
+    "Async support is particularly useful for calling multiple LLMs concurrently, as these calls are network-bound. Currently, only `OpenAI` is supported, but async support for other LLMs is on the roadmap.\n",
+    "\n",
+    "You can use the `agenerate` method to call an OpenAI LLM asynchronously."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "5e49e96c-0f88-466d-b3d3-ea0966bdf19e",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "I'm doing well. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "I am doing quite well. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing great, thank you! How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thanks for asking. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\u001b[1mConcurrent executed in 1.93 seconds.\u001b[0m\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing well, thank you. How about you?\n",
+      "\n",
+      "\n",
+      "I'm doing great, thank you. How about you?\n",
+      "\u001b[1mSerial executed in 10.54 seconds.\u001b[0m\n"
+     ]
+    }
+   ],
+   "source": [
+    "import time\n",
+    "import asyncio\n",
+    "\n",
+    "from langchain.llms import OpenAI\n",
+    "\n",
+    "def generate_serially():\n",
+    "    llm = OpenAI(temperature=0.9)\n",
+    "    for _ in range(10):\n",
+    "        resp = llm.generate([\"Hello, how are you?\"])\n",
+    "        print(resp.generations[0][0].text)\n",
+    "\n",
+    "\n",
+    "async def async_generate(llm):\n",
+    "    resp = await llm.agenerate([\"Hello, how are you?\"])\n",
+    "    print(resp.generations[0][0].text)\n",
+    "\n",
+    "\n",
+    "async def generate_concurrently():\n",
+    "    llm = OpenAI(temperature=0.9)\n",
+    "    tasks = [async_generate(llm) for _ in range(10)]\n",
+    "    await asyncio.gather(*tasks)\n",
+    "\n",
+    "\n",
+    "s = time.perf_counter()\n",
+    "# If running this outside of Jupyter, use asyncio.run(generate_concurrently())\n",
+    "await generate_concurrently() \n",
+    "elapsed = time.perf_counter() - s\n",
+    "print('\\033[1m' + f\"Concurrent executed in {elapsed:0.2f} seconds.\" + '\\033[0m')\n",
+    "\n",
+    "s = time.perf_counter()\n",
+    "generate_serially()\n",
+    "elapsed = time.perf_counter() - s\n",
+    "print('\\033[1m' + f\"Serial executed in {elapsed:0.2f} seconds.\" + '\\033[0m')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/modules/llms/how_to_guides.rst

@@ -7,6 +7,7 @@ They are split into two categories:
 
 1. `Generic Functionality <./generic_how_to.html>`_: Covering generic functionality all LLMs should have.
 2. `Integrations <./integrations.html>`_: Covering integrations with various LLM providers.
+3. `Asynchronous <./async_llm.html>`_: Covering asynchronous functionality.
 
 .. toctree::
    :maxdepth: 1