Skip to content

Make plugins sandbox-agnostic #2101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 55 commits into from
Jun 20, 2024
Merged

Make plugins sandbox-agnostic #2101

merged 55 commits into from
Jun 20, 2024

Conversation

Shimada666
Copy link
Contributor

@Shimada666 Shimada666 commented May 27, 2024

#1979

The goal is to decouple the sandbox from the plugins, allowing the plugins to run in most sandbox images.
Using a miniforge isolated environment, mambarequirement is responsible for installing miniforge and pip/c library.
After testing, The agent_skill and jupyter plugins can run on an Ubuntu18/Debian12 image with only sshd and sudo installed.

@Shimada666
Copy link
Contributor Author

Current progress: Support for Debian/Ubuntu operating systems has been added. Now it is possible to freely specify the sandbox_container_image. The sandbox will automatically install sshd when the program starts. sshd installation will be completed in about 30 seconds, followed by the installation of plugins. Finally, the fake shell will appear.

Although there are some minor bugs and some ugly code, it is already barely usable.

15:15:20 - opendevin:INFO: ssh_box.py:211 - SSHBox is running as root user with USER_ID=502 in the sandbox
15:15:21 - opendevin:INFO: ssh_box.py:235 - aorwall/swe-bench-sympy_sympy-testbed:1.0
15:15:21 - opendevin:INFO: ssh_box.py:671 - Container stopped
15:15:21 - opendevin:WARNING: ssh_box.py:683 - Using port forwarding for Mac OS. Server started by OpenDevin will not be accessible from the host machine at the moment. See https://github.com/OpenDevin/OpenDevin/issues/897 for more information.
15:15:21 - opendevin:INFO: ssh_box.py:656 - Mounting workspace directory: /Users/xxx/WebstormProjects/OpenDevin/workspace
15:15:21 - opendevin:INFO: ssh_box.py:692 - Mounting volumes: {'/Users/xxx/WebstormProjects/OpenDevin/workspace': {'bind': '/workspace', 'mode': 'rw'}, '/tmp/cache': {'bind': '/root/.cache', 'mode': 'rw'}}
15:15:21 - opendevin:INFO: ssh_box.py:656 - Mounting workspace directory: /Users/xxx/WebstormProjects/OpenDevin/workspace
15:15:21 - opendevin:INFO: ssh_box.py:705 - Container started
15:15:22 - opendevin:INFO: ssh_box.py:721 - waiting for container to start: 1, container status: running
15:15:23 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:24 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:29 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:29 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:34 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:34 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:39 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:39 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:44 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:44 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:49 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:49 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:15:54 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:15:55 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:16:00 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:16:00 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:16:05 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:16:05 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:16:10 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:16:10 - opendevin:ERROR: ssh_box.py:405 - Failed to login to SSH session, retrying...
15:16:15 - opendevin:INFO: ssh_box.py:398 - Connecting to root@localhost via ssh. If you encounter any issues, you can try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b' and report the issue on GitHub. If you started OpenDevin with `docker run`, you should try `ssh -v -p 61684 root@localhost` with the password '0467fe37-7b34-4b42-8f95-365beba6de3b on the host machine (where you started the container).
15:16:17 - opendevin:INFO: ssh_box.py:754 - Interactive Docker container started. Type 'exit' or use Ctrl+C to exit.
15:16:18 - opendevin:INFO: mixin.py:34 - Copied files from [/Users/xxx/WebstormProjects/OpenDevin/opendevin/runtime/plugins/mamba] to [/opendevin/plugins/mamba] inside sandbox.
15:16:18 - opendevin:INFO: mixin.py:42 - Initializing plugin [mamba] by executing [/opendevin/plugins/mamba/setup.sh] in the sandbox.
15:17:10 - opendevin:INFO: mixin.py:57 - Plugin mamba initialized successfully
15:17:10 - opendevin:INFO: mixin.py:34 - Copied files from [/Users/xxx/WebstormProjects/OpenDevin/opendevin/runtime/plugins/agent_skills] to [/opendevin/plugins/agent_skills] inside sandbox.
15:17:10 - opendevin:INFO: mixin.py:42 - Initializing plugin [agent_skills] by executing [/opendevin/plugins/agent_skills/setup.sh] in the sandbox.
15:17:29 - opendevin:INFO: mixin.py:57 - Plugin agent_skills initialized successfully
15:17:30 - opendevin:INFO: mixin.py:34 - Copied files from [/Users/xxx/WebstormProjects/OpenDevin/opendevin/runtime/plugins/jupyter] to [/opendevin/plugins/jupyter] inside sandbox.
15:17:30 - opendevin:INFO: mixin.py:42 - Initializing plugin [jupyter] by executing [/opendevin/plugins/jupyter/setup.sh] in the sandbox.
15:18:08 - opendevin:INFO: mixin.py:57 - Plugin jupyter initialized successfully
15:18:09 - opendevin:INFO: mixin.py:71 - Sourced ~/.bashrc successfully
15:18:09 - opendevin:INFO: ssh_box.py:764 - --- AgentSkills COMMAND DOCUMENTATION ---
open_file(path: str, line_number: Optional[int] = None) -> None:
    Opens the file at the given path in the editor. If line_number is provided, the window will be moved to include that line.
    Args:
    path: str: The path to the file to open.
    line_number: Optional[int]: The line number to move to.

goto_line(line_number: int) -> None:
    Moves the window to show the specified line number.
    Args:
    line_number: int: The line number to move to.

scroll_down() -> None:
    Moves the window down by 100 lines.
    Args:
    None

scroll_up() -> None:
    Moves the window up by 100 lines.
    Args:
    None

create_file(filename: str) -> None:
    Creates and opens a new file with the given name.
    Args:
    filename: str: The name of the file to create.

edit_file(start: int, end: int, content: str) -> None:
    Edit a file.
    It replaces lines `start` through `end` (inclusive) with the given text `content` in the open file. Remember, the file must be open before editing.
    Args:
    start: int: The start line number. Must satisfy start >= 1.
    end: int: The end line number. Must satisfy start <= end <= number of lines in the file.
    content: str: The content to replace the lines with.

search_dir(search_term: str, dir_path: str = './') -> None:
    Searches for search_term in all files in dir. If dir is not provided, searches in the current directory.
    Args:
    search_term: str: The term to search for.
    dir_path: Optional[str]: The path to the directory to search.

search_file(search_term: str, file_path: Optional[str] = None) -> None:
    Searches for search_term in file. If file is not provided, searches in the current open file.
    Args:
    search_term: str: The term to search for.
    file_path: Optional[str]: The path to the file to search.

find_file(file_name: str, dir_path: str = './') -> None:
    Finds all files with the given name in the specified directory.
    Args:
    file_name: str: The name of the file to find.
    dir_path: Optional[str]: The path to the directory to search.

parse_pdf(file_path: str) -> None:
    Parses the content of a PDF file and prints it.
    Args:
    file_path: str: The path to the file to open.

parse_docx(file_path: str) -> None:
    Parses the content of a DOCX file and prints it.
    Args:
    file_path: str: The path to the file to open.

parse_latex(file_path: str) -> None:
    Parses the content of a LaTex file and prints it.
    Args:
    file_path: str: The path to the file to open.

parse_pptx(file_path: str) -> None:
    Parses the content of a pptx file and prints it.
    Args:
    file_path: str: The path to the file to open.


---
$ echo "print(1)" | execute_cli
15:18:35 - opendevin:INFO: ssh_box.py:790 - exit code: 0
15:18:35 - opendevin:INFO: ssh_box.py:791 - [Code executed successfully with no output]
15:18:35 - opendevin:INFO: ssh_box.py:794 - background logs: ...
$ echo "open_file('aa.py')" | execute_cli
15:18:52 - opendevin:INFO: ssh_box.py:790 - exit code: 0
15:18:52 - opendevin:INFO: ssh_box.py:791 - [File: /workspace/aa.py (2 lines total)]
1|if 'aa' in None:
2|    print('bb')

15:18:52 - opendevin:INFO: ssh_box.py:794 - background logs: ..
$ echo "print(1)" | execute_cli
15:21:10 - opendevin:INFO: ssh_box.py:790 - exit code: 0
15:21:10 - opendevin:INFO: ssh_box.py:791 - 1

15:21:10 - opendevin:INFO: ssh_box.py:794 - background logs: ..............

Shimada666 and others added 3 commits June 6, 2024 00:40
# Conflicts:
#	opendevin/runtime/docker/ssh_box.py
#	opendevin/runtime/plugins/mixin.py
Copy link
Contributor

@neubig neubig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Shimada666 , this looks awesome, thanks for contributing! I'm happy to review when it's ready.

@Shimada666
Copy link
Contributor Author

@neubig Thank you very much. I feel a bit embarrassed; this PR should have been ready for review much earlier, but I got a bit busy before and it was delayed.😢 I will further improve this PR and have it ready for review by tomorrow (the day after at the latest). Thanks again!

@neubig
Copy link
Contributor

neubig commented Jun 8, 2024

No worries at all, thank you!

@Shimada666
Copy link
Contributor Author

Since the plugins depend on sshd for execution, we must include sshd in the image. In the previous example, I installed sshd in Ubuntu by specifying the entrypoint, and it successfully ran on Ubuntu 18.04/Debian 12, like this:

            self.container = self.docker_client.containers.run(
                self.container_image,
                entrypoint='bash',
                # allow root login
                user='root',
                command=f"-c \"apt update && apt install openssh-server;mkdir /var/run/sshd && mkdir -p -m0755 /var/run/sshd;/usr/sbin/sshd -D -p {self._ssh_port} -o 'PermitRootLogin=yes'\"",
                **network_kwargs,
                working_dir=self.sandbox_workspace_dir,
                name=self.container_name,
                detach=True,
                volumes=self.volumes,
            )

However, this approach is undoubtedly ugly and not universal. For instance, it doesn't apply to CentOS (which uses yum). Therefore, this PR does not include that part. Currently, it still requires an image with sshd built in to run, like ghcr.io/opendevin/sandbox:main.

After discussing with @xingyaoww, maybe we should abandon SSH and switch to using WebSocket communication. in future refactoring. A better approach is for users to provide a base image (CentOS/Ubuntu/Debian), from which we generate a Dockerfile to install the necessary runtime, rebuild the sandbox image, and then use this image.

So, I plan for this PR to be used only for isolating the Python environment for opendevin and creating image caches to speed up startup times. When use_cache is set to true in config.toml, the sandbox cache image will be automatically generated after the first run, and this cache image will be used in subsequent runs to reduce the startup time of the sandbox. You can test it with the following command

config.toml

[core]
...
use_cache = false
sandbox_container_image = "ghcr.io/opendevin/sandbox:main"

run command

python3 opendevin/runtime/docker/ssh_box.py

Currently, I am using the Python base environment (3.10) provided by miniforge as the runtime for opendevin. If needed, it is also easy to install Python 3.11 and use it as the runtime, with the trade-off being a slightly slower startup time the first time it is launched.

@Shimada666 Shimada666 marked this pull request as ready for review June 9, 2024 06:45
Copy link
Collaborator

@xingyaoww xingyaoww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM! I think we can remove use_cache and the logic for docker commit for now, and aim for a better Dockerfile-based build solution directly and transparent to the user.

@xingyaoww
Copy link
Collaborator

We can maybe directly create this "Dockefile builder" and put dependencies of mamba and sshd directly into it to make it sandbox-agnostic (for debian) - then in my next architectural change PR, i can get rid of the dependency on SSHD.

@Shimada666
Copy link
Contributor Author

Ok, I will try to refactor the image building logic into code so that we don't need the mamba plugin!

@Shimada666 Shimada666 requested a review from xingyaoww June 9, 2024 10:16
@Shimada666
Copy link
Contributor Author

I have added support for automatically building the agnostic_sandbox, and I have tested it on Ubuntu 22.04/Debian 11, and it works fine! To test, simply set sandbox_container_image = "ubuntu:22.04" in config.toml.

@Shimada666
Copy link
Contributor Author

Now all CI tests are passed, I am ready to be reviewed again!

@tobitege
Copy link
Collaborator

tobitege commented Jun 19, 2024

Now all CI tests are passed, I am ready to be reviewed again!

I can't review all the sandbox topics myself, I'll leave that to others.
But still would like to ask to only cause downloads if tests are run in CI on the server.

Maybe add a test fixture .skipif (see other tests) that could skip the test if TEST_IN_CI is not true?
You may find examples in the test_agent.py for that.

Example:

@pytest.mark.skipif(os.getenv('TEST_IN_CI') != 'true',
    reason='Only run on CI',
)

@Shimada666
Copy link
Contributor Author

@tobitege Got it! I have added the skipif, please take a look.

@tobitege
Copy link
Collaborator

@tobitege Got it! I have added the skipif, please take a look.

Thanks! Workflow is running.

@tobitege tobitege requested a review from li-boxuan June 19, 2024 07:42
Copy link
Collaborator

@xingyaoww xingyaoww left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally, all tests passed - LGTM. Happy to wait a bit if someone else want to take a look before merge.


pip install flake8 python-docx PyPDF2 python-pptx pylatexenc openai opencv-python
source ~/.bashrc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is this line necessary here now we are source script at every plugin initialization? Bc. pip install probably don't require the PYTHONPATH?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it will be necessary. I plan to make improvements in the next PR by removing anything unnecessary.

@Shimada666
Copy link
Contributor Author

@tobitege @yufansong @li-boxuan
Hi, anyone can take a look for this PR again? I hope the PR can be merged, then I can proceed with the next optimizations, including reducing the image size, removing unnecessary dependencies and code! 😄

Copy link
Collaborator

@li-boxuan li-boxuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand this PR but I don't have any concern either.

_test_sandbox_jupyter_agentskills_fileop_pwd_impl(box)


@pytest.mark.skipif(os.getenv('TEST_IN_CI') != 'true',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work because you are not adding TEST_IN_CI=true env variable to test-for-sandbox in ghcr.yml.

But don't worry, let me push a separate commit once this PR is merged. Not a big deal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a big thanks!

@li-boxuan li-boxuan merged commit 26fc3c8 into All-Hands-AI:main Jun 20, 2024
2 checks passed
@tobitege
Copy link
Collaborator

@Shimada666 I think we need some more testing of this PR.
From current main branch, I run make build and make run.
If this matters, in my config.toml I switched to sandbox_container_image = "ghcr.io/opendevin/sandbox:0.6.2" (as on the website).

Now seeing this in the terminal log (WSL) in OpenDevin starting a new session in browser:

09:07:55 - opendevin:INFO: ssh_box.py:743 - Container started
09:07:56 - opendevin:INFO: ssh_box.py:759 - waiting for container to start: 1, container status: running
09:07:56 - opendevin:ERROR: session.py:69 - Error creating controller: An error occurred while checking if miniforge3 directory exists: b''
Traceback (most recent call last):
  File "/mnt/d/github/OpenDevin/opendevin/server/session/session.py", line 67, in _initialize_agent
    await self.agent_session.start(data)
  File "/mnt/d/github/OpenDevin/opendevin/server/session/agent.py", line 46, in start
    await self._create_runtime()
  File "/mnt/d/github/OpenDevin/opendevin/server/session/agent.py", line 65, in _create_runtime
    self.runtime = ServerRuntime(self.event_stream, self.sid)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/d/github/OpenDevin/opendevin/runtime/server/runtime.py", line 35, in __init__
    super().__init__(event_stream, sid, sandbox)
  File "/mnt/d/github/OpenDevin/opendevin/runtime/runtime.py", line 73, in __init__
    self.sandbox = create_sandbox(sid, config.sandbox_type)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/d/github/OpenDevin/opendevin/runtime/runtime.py", line 47, in create_sandbox
    return DockerSSHBox(sid=sid)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/d/github/OpenDevin/opendevin/runtime/docker/ssh_box.py", line 263, in __init__
    self.setup_user()
  File "/mnt/d/github/OpenDevin/opendevin/runtime/docker/ssh_box.py", line 360, in setup_user
    raise Exception(
Exception: An error occurred while checking if miniforge3 directory exists: b''

Any ideas?

@Shimada666
Copy link
Contributor Author

@tobitege You need use ghcr.io/opendevin/sandbox:main! ghcr.io/opendevin/sandbox:0.6.2 can't work because it's without miniforge3 python

@tobitege
Copy link
Collaborator

So this also applies to the main README then which still states 0.6.2?

@tobitege
Copy link
Collaborator

However, the exception ideally should be handled by adding a check for miniforge3 being present and if not then output a transparent error message to the user, imho, to tell the user what else to do.
Would that be an acceptable addition?

@Shimada666
Copy link
Contributor Author

It was there before. I just found out it was removed by this PR. I think it should be added back.
#2538

@Shimada666
Copy link
Contributor Author

@tobitege
Maybe we should revert #2538 ?

@tobitege
Copy link
Collaborator

tobitege commented Jun 21, 2024

@tobitege Maybe we should revert #2538 ?

Edit: approved #2560

@Shimada666
Copy link
Contributor Author

#2560
I create a revert PR, please take a look! @tobitege @SmartManoj

@tobitege
Copy link
Collaborator

tobitege commented Jun 21, 2024

@Shimada666 one more thing, does the Makefile also need a change (right at the top):

It currently has:
DOCKER_IMAGE = ghcr.io/opendevin/sandbox
Should that also be using :main?
DOCKER_IMAGE = ghcr.io/opendevin/sandbox:main

Otherwise the log shows that docker is pulling latest during make build:

Pulling Docker image...
Using default tag: latest
latest: Pulling from opendevin/sandbox
Digest: sha256:4bd05c581692e26a448bbc6771a21bb27002cb0e6bcf5034d0db91bb8704d0f0
Status: Image is up to date for ghcr.io/opendevin/sandbox:latest
ghcr.io/opendevin/sandbox:latest

Reference: #2561

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants