Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #7473: Prevent OpenHands from creating additional pull requests unless explicitly instructed #7476

Merged
merged 8 commits into from
Mar 27, 2025

Conversation

xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Mar 24, 2025

This PR addresses issue #7473 by adding explicit instructions to the system prompt and GitHub microagent to prevent OpenHands from creating multiple pull requests for a single issue/session.

Changes

  1. Added a new <GITHUB_PULL_REQUESTS> section to the system prompt with clear instructions to create only ONE pull request per session/issue unless explicitly instructed otherwise.
  2. Updated the GitHub microagent to reinforce this behavior with more detailed instructions.
  3. Added tests to verify that the system prompt and GitHub microagent contain the new instructions.

Testing

Added unit tests that verify the system prompt and GitHub microagent contain the new instructions.

@xingyaoww can click here to continue refining the PR


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:f20a794-nikolaik   --name openhands-app-f20a794   docker.all-hands.dev/all-hands-ai/openhands:f20a794

</VERSION_CONTROL>

<PULL_REQUESTS>
* Create only ONE pull request per session/issue unless explicitly instructed otherwise.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small question here, I wonder if this could affect evals. It seems the new instructions on git say "do this", while the version control section are all "when", "if", "do not".

Feel free to ignore if you don't think so.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new instructions on git say "do this",

You mean the github.md file?

I assume all "git" relate issue probably won't affect eval -- we are just doing git add -A && git diff {base_commit} XXX on eval stuff so should be able to capture ALL diff 🤔

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not github.md, I mean this new section PULL_REQUESTS is in the system prompt, and it tells the LLM "create one PR per session/issue".

I wonder if the LLM will try to "create one PR", and spend steps trying and failing and idk, worst case losing track.

The older sections, for example VERSION_CONTROL, never tells "do this", it says "when you commit... " (so if you never commit, it's a no-op).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very good point! i'll tweak that

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated! feel free to re-review 🙏

Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the PR will work as expected together with the microagent, but we could try this.

Had one more question on PRs, though I'm not sure it matters.

Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it!

@xingyaoww xingyaoww merged commit d69e4fe into main Mar 27, 2025
16 checks passed
@xingyaoww xingyaoww deleted the fix-github-pr-creation branch March 27, 2025 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants