Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Agent, LLM] Make sure codeact agent produce message in u/a/u/a order #3193

Merged
merged 7 commits into from
Aug 1, 2024
Merged

[Agent, LLM] Make sure codeact agent produce message in u/a/u/a order #3193

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3b657d4
make sure codeact agent produce message in u/a/u/a order
xingyaoww Jul 31, 2024
b7ce92b
Merge branch 'main' into xw/keep-ua-order
xingyaoww Jul 31, 2024
0fe13c9
Merge branch 'main' into xw/keep-ua-order
xingyaoww Jul 31, 2024
ad05fcd
integration tests
enyst Jul 31, 2024
1f13b47
Merge branch 'xw/keep-ua-order' of github.com:OpenDevin/OpenDevin int…
enyst Jul 31, 2024
e6f1787
sync message changes to codeact swe
xingyaoww Aug 1, 2024
ab90612
fix integration tests
xingyaoww Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion agenthub/codeact_agent/codeact_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,13 @@ def _get_messages(self, state: State) -> list[dict[str, str]]:

# add regular message
if message:
messages.append(message)
# handle error if the message is the SAME role as the previous message
# litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'detail': 'Only supports u/a/u/a/u...'}
# there should not have two consecutive messages from the same role
if messages and messages[-1]['role'] == message['role']:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't test, but it looks like this will also merge the example message (in_context_sample) with the first user message. If that's the case, will it have some effect on evals, since the first user message is the task?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the original implementation of CodeAct is actually merging this two message together - then it get separated at some moment. I think we had a "-- end of example --" message there, so i think we should be fine?

Copy link
Collaborator

@enyst enyst Jul 31, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely possible, it looks separated, and we have also a hard-coded "NOW, LET'S START!" after that.

Sorry, I'm running SWE-bench at the moment on another branch, as soon as it's done I can try this a little bit. But I think merging this is okay before, it doesn't have to wait. Up to you, this is a small thing we can tweak if needed, IMHO.

messages[-1]['content'] += '\n\n' + message['content']
else:
messages.append(message)

# the latest user message is important:
# we want to remind the agent of the environment constraints
Expand Down