feat: husarion toolset #41
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fix: append human message if agent returns an aimsg without a tool call (hacky) fix: robot_y calculation bug
There was a problem hiding this comment.
Thank you for this feature!
Some general comments after testing with O3DE simulation:
- I think agents get mislead by the SetWaypointTool and sometimes uses it as a goal pose tool
- we really need handle navigation through actions - I noticed problems with:
- nav stack not receiving the goal_pose, even though it was published
- robot is stuck and can't make progress, but llm gives the same goal pose
I think agent even though it identifies objects in the first frame, then it don't use SetWaypointTool any more, and seem not to use its output.
boczekbartek
approved these changes
Jun 27, 2024
boczekbartek
requested changes
Jun 27, 2024
boczekbartek
approved these changes
Jun 27, 2024
There was a problem hiding this comment.
Tested again with o3de simulation and works well. I attach output files:
map_database.txt
history.zip
boczekbartek
pushed a commit
that referenced
this pull request
Jul 2, 2024
* build: specify flake8 version * feat: add transform listener * refactor: use /tf for getting current pose * feat: wip husarion the describer demo example * feat: husarion the descriptor wip * refactor: rename tools.ros.cli_tools to tools.ros.cli fix: append human message if agent returns an aimsg without a tool call (hacky) fix: robot_y calculation bug * style: adjust too long line * refactor: rename SetWaypointTool to AddDescribedWaypointToDatabaseTool * refactor: set waypoint tool * style: use keyword arguments for cv2 functions * misc: update tool docstring, rename example * fix: remove -v flag as not all ros2 topic commands support it * docs: document new example
boczekbartek
pushed a commit
that referenced
this pull request
Aug 5, 2024
* build: specify flake8 version * feat: add transform listener * refactor: use /tf for getting current pose * feat: wip husarion the describer demo example * feat: husarion the descriptor wip * refactor: rename tools.ros.cli_tools to tools.ros.cli fix: append human message if agent returns an aimsg without a tool call (hacky) fix: robot_y calculation bug * style: adjust too long line * refactor: rename SetWaypointTool to AddDescribedWaypointToDatabaseTool * refactor: set waypoint tool * style: use keyword arguments for cv2 functions * misc: update tool docstring, rename example * fix: remove -v flag as not all ros2 topic commands support it * docs: document new example
adamdbrw
pushed a commit
that referenced
this pull request
Aug 14, 2024
* build: specify flake8 version * feat: add transform listener * refactor: use /tf for getting current pose * feat: wip husarion the describer demo example * feat: husarion the descriptor wip * refactor: rename tools.ros.cli_tools to tools.ros.cli fix: append human message if agent returns an aimsg without a tool call (hacky) fix: robot_y calculation bug * style: adjust too long line * refactor: rename SetWaypointTool to AddDescribedWaypointToDatabaseTool * refactor: set waypoint tool * style: use keyword arguments for cv2 functions * misc: update tool docstring, rename example * fix: remove -v flag as not all ros2 topic commands support it * docs: document new example
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.