feat: add simple rclpy based function calling

[boczekbartek]

• adds rclpy based function calling
• simple random goal pose example

[maciejmajek]

Thank you for the feature.
Few things.
rfc: I think that naming the file like the rclpy python package might be a little bit confusing at some point, especially for newcomers

[maciejmajek]

Even though this function does not need an input, please use the BaseModel and BaseTool for tool construction.
I've found out why the cli tool works better for discovering the topics. Do you think we can build something like that? link

[boczekbartek]

Why additional boilerplate is required for a simple tool?

Do you think we can build something like that?

We can try, but how about in another PR?

When you launch any ros2 command-line tool the first time, a daemon is spun up in the background.

I would have to research more what they mean by the daemon and how to run it with rclpy

[maciejmajek]

Because in the future we may use an output parser that requires specific input format.
See for example how openai agent is built by default (tools (Sequence[BaseTool]))
link

output parser

We can try, but how about in another PR?

I'd rather we include this functionality in this pr (especially if the code below works well). We are iterating really fast, I think that we may start to introduce a more finalized parts of the code without extending the tech debt

Consider the following:

import rclpy
from rclpy.node import Node
from std_msgs.msg import String
import json

class DiscoveryNode(Node):
    def __init__(self):
        super().__init__('discovery_node')
        self.timer = self.create_timer(1.0, self.update_discovery)
        self.publisher = self.create_publisher(String, 'discovery_info', 10)
        self.topics = {}
        self.nodes = set()

    def update_discovery(self):
        self.topics = dict(self.get_topic_names_and_types())
        self.nodes = set(self.get_node_names())
        
        discovery_info = {
            'topics': self.topics,
            'nodes': list(self.nodes)
        }
        
        msg = String()
        msg.data = json.dumps(discovery_info)
        self.publisher.publish(msg)

class YourNode(Node):
    def __init__(self):
        super().__init__('your_node')
        self.subscription = self.create_subscription(
            String,
            'discovery_info',
            self.discovery_callback,
            10)
        self.discovery_info = None

    def discovery_callback(self, msg):
        self.discovery_info = json.loads(msg.data)
        # Now you can use self.discovery_info['topics'] and self.discovery_info['nodes']

def main(args=None):
    rclpy.init(args=args)
    
    discovery_node = DiscoveryNode()
    your_node = YourNode()
    
    try:
        rclpy.spin(discovery_node)
    finally:
        discovery_node.destroy_node()
        your_node.destroy_node()
        rclpy.shutdown()

if __name__ == '__main__':
    main()

[boczekbartek]

See for example how openai agent is built by default

@tool decorator creates either Tool or StructuredTool, which inherit from BaseTool so I think no problem here, isn't it?

Consider the following:

thank you! I'll try it out

output parser

Thank you for sharing! I'll dive a bit deeper into this topic, because at the first glance it seems like @tool should work well and let you know.

In general:
In case it turns out that using @tool is not limiting in the areas you mentioned, would you still rather use BaseTool inheritance approach, for example for consistency or code style? (personally, I prefer simpler code if possible) . Of course if it's limiting I'm fully convinced!

[boczekbartek]

Thank you for sharing! I'll dive a bit deeper into this topic, because at the first glance it seems like @tool should work well and let you know.

Hmm in general I think Input(BaseModel) != OutputParser(BaseModel), so @tool can be used using output parser.

Please check this example:

import rclpy

from rai.tools.ros.rclpy import get_topics_names_and_types_tool
from langchain.output_parsers import PydanticOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field, validator
from langchain_openai import ChatOpenAI

class Ros2TopicInfo(BaseModel):
    topic: str = Field(description="topic name")
    msg_type: str = Field(description="message type of the topic")

    @validator("topic")
    def topci_begins_with_namespace(cls, field):
        if field[0] != "/":
            raise ValueError("Badly formed topic - should start with the ros namespace")
        if field[-1] == "/":
            raise ValueError("Badly formed topic - can't end with '/'")
        return field

def main(args=None):

    rclpy.init(args=args)

    model = ChatOpenAI(model="gpt-4o")
    model.bind_tools([get_topics_names_and_types_tool])

    content="You are an autonomous agent. Your main goal is to fulfill the user's requests. "
    "Do not make assumptions about the environment you are currently in. "
    "Use the tooling provided to gather information about the environment."

    parser = PydanticOutputParser(pydantic_object=Ros2TopicInfo)

    prompt = PromptTemplate(
        template="Answer the user query.\n{format_instructions}\n{query}\n",
        input_variables=["query"],
        partial_variables={"format_instructions": parser.get_format_instructions()},
    )

    chain = prompt | model | parser

    r = chain.invoke({"query": content})
    print(r)
    
    rclpy.shutdown()

if __name__ == '__main__':
    main()

[maciejmajek]

In case it turns out that using @tool is not limiting in the areas you mentioned

Sorry, I still believe that declaring a tool via a decorator is limiting. However I agree, that it is not THAT much limiting in the mentioned areas.
Thank you for the research

would you still rather use BaseTool inheritance approach, for example for consistency or code style?

Well, I am a big fan of consistency and I definitely prefer to have all the tools in the same format, but since this is such a simple tool with no need for input we can leave it as it is.
Thank you

Did you reconsider adding a ros2 node for topic discovery?

[boczekbartek]

Did you reconsider adding a ros2 node for topic discovery?

As discussed with @adamdbrw today, I think the way to go is one master RAI node, not a separate discovery node.
I will add it to this PR soon

[boczekbartek]

@maciejmajek

I think that naming the file like the rclpy python package might be a little bit confusing at some point, especially for newcomers

I also considered:

1. ros_py
2. native (rai.tools.ros.native)

How about them?

[maciejmajek]

I think that native (rai.tools.ros.native) is 10/10

[boczekbartek]

@maciejmajek
Renamed to native in b65332e

[boczekbartek]

@maciejmajek Could you review again?

[maciejmajek]

Running the demo without husarion's topics causes some problems.

[maciejmajek]

One change please. Otherwise LGTM