Log tool_instances to database

[simonw]

Refs:

• Toolbox instantiation should be logged to DB when logging toolbox methods #1089

TODO:

• Tests for toolboxes in sync mode
• Tests for async toolbox methods in sync mode (async def list_files())
• Tests for toolboxes in async mode
• Tests for async toolbox methods in async mode

[simonw]

llm -f tests/test_plugins.py -f llm/models.py -m o3 -s \
  'The memory_set and memory_get test results look wrong, {"args": [], "kwargs": {}} should not happen, and {"name": "Memory_get", "output": "two", "instance": None} has instance None when it should be populated'

https://gist.github.com/simonw/2709b0b9b960fd13edad70b830f3509f - looks like it figured out the {"args": [], "kwargs": {}} bit for me:

def wrapped_init(self, *args, **kwargs):
    sig = inspect.signature(original_init)
    bound = sig.bind(self, *args, **kwargs)
    bound.apply_defaults()

    self._config = {
        name: value
        for name, value in bound.arguments.items()
        if name != "self"
        and sig.parameters[name].kind
        not in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD)
    }

    original_init(self, *args, **kwargs)

Got Claude to write me some tests to help exercise that: https://claude.ai/share/6153b936-4c50-4af4-a37b-72339bc01717

def test_toolbox_config_capture():
    """Test that Toolbox captures __init__ parameters in _config"""
    
    # Single positional arg
    class Tool1(Toolbox):
        def __init__(self, value):
            pass
    
    assert Tool1(42)._config == {"value": 42}
    
    # Multiple positional args
    class Tool2(Toolbox):
        def __init__(self, a, b, c):
            pass
    
    assert Tool2(1, 2, 3)._config == {"a": 1, "b": 2, "c": 3}
    
    # Keyword args with defaults
    class Tool3(Toolbox):
        def __init__(self, name="default", count=10):
            pass
    
    assert Tool3()._config == {"name": "default", "count": 10}
    assert Tool3(name="custom", count=20)._config == {"name": "custom", "count": 20}
    
    # Mixed args
    class Tool4(Toolbox):
        def __init__(self, required, optional="default"):
            pass
    
    assert Tool4("hello")._config == {"required": "hello", "optional": "default"}
    assert Tool4("world", optional="custom")._config == {"required": "world", "optional": "custom"}
    
    # Var args excluded
    class Tool5(Toolbox):
        def __init__(self, regular, *args, **kwargs):
            pass
    
    assert Tool5("test", 1, 2, extra="value")._config == {"regular": "test"}
    
    # No init
    class Tool6(Toolbox):
        pass
    
    assert Tool6()._config == {}

They fail against the original version but pass with o3's new version, so I'm going with it.

[simonw]

Test failures are because this feature was added in SQLite 2023-11-01 (3.44.0):

Aggregate functions can now include an ORDER BY clause after their last parameter

Impressed that Claude could tell me that, and even got the version it was added correct: https://claude.ai/share/22891a1f-8d79-40e3-8363-bb7b50bf53e6

And it wrote me a new query which seems to work.