Tool library

[bwachter]

There's not much related to gptel-tool use out there, part from the org tools. I recently realised it'd be useful to have an easily accessible library of common tools as any toy project playing with LLM tools needs at least some buffer manipulation functions.

Originally I was thinking about doing a pull request to this repo - but I don't think it fits in here. So, I ended up publishing most of my well working tools at https://github.com/aard-fi/gptel-tool-library and added a wrapper library for easy loading/unloading of tool groups. Mentioning that (plus https://git.bajsicki.com/phil/gptel-org-tools - which helped me a lot to get started) in the section about tool use might be useful for others as well.

[karthink]

Have you seen https://github.com/skissue/llm-tool-collection?

This is intended to be the canonical repo for collecting user-created tools for gptel and llm (which share a common tool format).

I suggest coordinating with @skissue and contributing to a single repo to avoid confusion and fragmentation.

[bwachter]

No, I wasn't aware of that. I went through that code now, and it doesn't look like our two different approaches to the same problem are compatible.

[bajsicki]

Short clarification, the repo moved to https://codeberg.org/bajsicki/gptel-got :)

I don't think it's very easy to coordinate, because everyone's use-case is different, and between different models, different approaches and needs... I think we'll need to develop a standard for the tooling, and some way of managing these tools.

I believe gptel's presets are a good start to that end.

[bwachter]

I don't think it's very easy to coordinate, because everyone's use-case is different, and between different models, different approaches and needs... I think we'll need to develop a standard for the tooling, and some way of managing these tools.

It's already good that llm and gptel have the same tool format (and same style accessor functions) - I wasn't aware of that before. After that it already gets tricky:

llm-tool-collection tries to have everything in one place, and provide a macro for tool definition to handle the various llms.

My approach instead is just having the core library a loader, without any tools defined, and topic specific tools defined in their own module. For the tool definition I prefer to just use the make-tool functions, and have function definitions separate - I will probably expand that to provide a wrapper that calls all the available make tool functions, but I don't want more abstraction than that. I also currently have tools sorted into three categories, based on their safety, so that I can bundle all of that in one module, and give the user the choice how safe (or unsafe) tools they want to use.

Overall my focus is just on simple loading/unloading of tool libraries following a few requirements - which also allows other libraries to easily bundle bindings tightly coupled with their api. That was my main motivation for getting started on this - I'm using that for AI controlled turtle graphis and AI arch installer interface

[bajsicki]

Hooking this into gptel to feed the tools in sequence based on the LLM's output (e.g. regexp on the content response for a 'confidence score') would be great, since then we can let them follow a procedure more accurately.

E.g.

1. Tool set A + request -> LLM
2. Does LLM response pass regexp test?

If yes -> feed different tool set and further steps
If no -> request to LLM with steps to refine the result

I can totally see this kind of workflow streamlining an LLM's ability to find accurate data across org.

[tachyonicbytes]

Hi! I have a small question that I don't think warrants a new github issue. @karthink, in your videos you have a tool called search_elfeed_db, that's not shared, as far as I can tell, in any place that contains gptel tools. Would you be willing to share it?

I am trying to obtain the interface present in this article: https://koomen.dev/essays/horseless-carriages/, and Emacs seems well poised for this, both for RSS feeds and potentially email filters.

[karthink]

Short clarification, the repo moved to https://codeberg.org/bajsicki/gptel-org :)

I think we discussed this when we spoke, but the feature name gptel-org is taken by gptel. If the problem really is just that you don't want to be typing the gptel-org-tools- prefix all the time, you can define a read-symbol shorthand. Add the file-local variable read-symbol-shorthands with the value (("got-" . "gptel-org-tools-")) to your elisp file, and you can type got- instead.

EDIT: Another approach, and the one I used because typing org-latex-preview- got tedious, was to define an elisp-mode abbrev so that olp expanded to org-latex-preview.

[karthink]

Hooking this into gptel to feed the tools in sequence based on the LLM's output (e.g. regexp on the content response for a 'confidence score') would be great

The new package dsel works like this, although it uses llm under the hood, not gptel.

Doing something like this with gptel is quite easy, though there is no declarative interface.

I'm not sure how this topic connects to this thread, BTW.

[karthink]

In your videos you have a tool called search_elfeed_db, that's not shared, as far as I can tell, in any place that contains gptel tools. Would you be willing to share it?

@tachyonicbytes https://paste.karthinks.com/f6924d61-gptel-local-search-tools.el.html

This tool was for demonstration purposes only. Using regular keyword searches doesn't really help with retrieval, since I can do that faster and more comfortably myself than asking an LLM. So the tool is not useful.

Similarity search on the Elfeed (etc) database would be more useful, but it's not easy to set that up. Elfeed entry content isn't even searchable.

[karthink]

Converting to a discussion as there is no action needed in gptel.