Merge pull request #57 from eli5-org/oss-llm-references

Add more references to open source libraries for LLM explainability

Author: lopuhin

docs/source/_notebooks/explain_llm_logprobs.rst

@@ -669,3 +669,11 @@ Then you can explain predictions with a custom client:
 
 
 
+
+Consider checking other libraries which support explaining predictions
+of open source LLMs:
+
+- `optillm <https://github.com/codelion/optillm>`__, e.g. see
+  `codelion/optillm#168 <https://github.com/codelion/optillm/discussions/168#discussioncomment-12399569>`__
+- you can also visualize the ouputs using the `logprobs
+  visualizer <https://huggingface.co/spaces/codelion/LogProbsVisualizer>`__

docs/source/libraries/openai.rst

@@ -57,11 +57,20 @@ you may call :func:`eli5.explain_prediction` with
 See the :ref:`tutorial <explain-llm-logprobs-tutorial>` for a more detailed usage
 example.
 
+Consider also checking other libraries which support explaining predictions of open source LLMs:
+
+- `optillm <https://github.com/codelion/optillm>`_, e.g. see
+  `codelion/optillm#168 <https://github.com/codelion/optillm/discussions/168#discussioncomment-12399569>`_
+- you can also visualize the ouputs using the
+  `logprobs visualizer <https://huggingface.co/spaces/codelion/LogProbsVisualizer>`_
+
 .. note::
     While token probabilities reflect model uncertainty in many cases,
     they are not always indicative,
     e.g. in case of `Chain of Thought <https://arxiv.org/abs/2201.11903>`_
     preceding the final response.
+    See the :ref:`tutorial's limitations section <explain-llm-logprobs-tutorial>`
+    for an example of that.
 
 .. note::
     Top-level :func:`eli5.explain_prediction` calls are dispatched

notebooks/explain_llm_logprobs.ipynb

@@ -751,6 +751,17 @@
     "    model=\"mlx-community/Mistral-7B-Instruct-v0.3-4bit\",\n",
     ")"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cb6fce33-edb1-42ab-ac93-7bd4472075ca",
+   "metadata": {},
+   "source": [
+    "Consider checking other libraries which support explaining predictions of open source LLMs:\n",
+    "\n",
+    "- [optillm](https://github.com/codelion/optillm), e.g. see [codelion/optillm#168](https://github.com/codelion/optillm/discussions/168#discussioncomment-12399569)\n",
+    "- you can also visualize the ouputs using the [logprobs visualizer](https://huggingface.co/spaces/codelion/LogProbsVisualizer)"
+   ]
   }
  ],
  "metadata": {