[Arch] Shrink runtime image size #3051

xingyaoww · 2024-07-20T20:54:51Z

What is the problem that this fixes or functionality that this introduces? Does it fix any open issues?

Previously, the runtime image was too large (~20+G) and caused a lot of headaches for deployment. @tobitege did a great job shrink the size in #3040.

Give a summary of what the PR does, explaining any non-trivial design decisions

This PR moves these sandbox image shrinking commits out of #3040 and tries to fix some existing issues. This will better prepare for an automated CI to build the Runtime image.

Other references

yufansong · 2024-07-21T03:16:58Z

opendevin/runtime/utils/runtime_build.py

+    # THIS blows up the image by 13 GB!!!
+    # Install (or update) the dependencies
+    # dockerfile_content += (
+    #     'RUN cd /opendevin/code && \\\n'


Is this used result from mamba related package?

Well, I actually made a mistake, it's not only 13 GB, but more like 22 GB on top.
The resulting sandbox was ~ 26GB. And most of that happens when mamba is installing and running poetry.
Idk what is happening there, but it ain't good. 😁

add mamba clean

tobitege · 2024-07-21T16:38:08Z

Some snippet that may help in _generate_dockerfile:

        dockerfile_content += (
            'ENV DEBIAN_FRONTEND=noninteractive '
            'POETRY_NO_INTERACTION=1 '
            'POETRY_VIRTUALENVS_IN_PROJECT=1 '
            'POETRY_VIRTUALENVS_CREATE=1 '
            'POETRY_CACHE_DIR=/tmp/poetry_cache\n'
        )

The above are used in other dockerfiles and they e.g. assure that poetry install --only main actually creates the virtualenv during image build, which fixes the "toml not found" error we chatted about in Slack.
Also, I had mistakenly removed that flag --only main in my attempts and that alone apparently is the reason for the image size going nuts because it just installed way too much unneeded stuff, doh! 😂

xingyaoww · 2024-07-21T16:39:40Z

@tobitege I think i finally got it working (at least locally). I did not do anything extra, but just remove that line of rm -rf /root/.cache/poetry 🤣

tobitege · 2024-07-21T16:41:26Z

@tobitege I think i finally got it working (at least locally). I did not do anything extra, but just remove that line of rm -rf /root/.cache/poetry 🤣

I'll send Sonnet the bill then lolol
Glad it works (again), my bad 😔

tobitege · 2024-07-21T17:00:06Z

I've experimented a little bit more and updated my PR's sources.
Marked my PR as experimental and draft now, to be safe.

You might take another peek at:
https://github.com/OpenDevin/OpenDevin/blob/d5ba379599421a5b119a9db83730719830dcbde6/opendevin/runtime/utils/runtime_build.py

added new method _generate_dockerfile_nodejs
you can try that locally with this command line after you pulled the image from the Docker hub manually (I haven't coded that in):
poetry run python ./opendevin/runtime/utils/runtime_build.py --base_image "nikolaik/python-nodejs:python3.11-nodejs21"
That image has Python 3.11, poetry, pip and NodeJS pre-installed. Final imagesize ~6GB.
For it to work, I had to apply a small change to _get_new_image_name before the final return:

        # Check if the base image already has the 'od_runtime' prefix
        if repo.startswith('od_runtime'):
            return f'{repo}:{tag}'

The way the above "nodejs" method is coded is that it doesn't require all the mamba stuff, so commands could just be run via poetry run ..., but I'm aware that mamba is used in several areas, like the Jupyter plugin.

I'll just wanted to share this as "food for thought", even though experimental code. 😀

tobitege · 2024-07-21T17:43:48Z

Btw, I think there's one big thing missing in the source copying: the agenthub folder isn't copied into the sandbox for dev updates? 🤔

xingyaoww · 2024-07-21T17:44:11Z

@tobitege this itself should be ready for review if it is ok with you!

I'm experimenting with a slightly more general/cleaner way of building OpenDevin runtime images (e.g., using Jinja templates), which hopefully can provide a cleaner way to deal with these docker files in Python, and make it easier to extend these dockerfiles to different languages (e.g., nodejs).

The way the above "nodejs" method is coded is that it doesn't require all the mamba stuff

The only previous reason for including mamba is that it can manage multiple versions of Python easily and can work out of the box when the base image does not contain the right Python version (e.g., py3.11), which is not your case for ubuntu+py3.11, though.

I guess if we get the Jinja template working nicely and cleanly, we could just start building every image with ubuntu+py3.11 and get rid of Mamba completely to save space, and using the mamba workflow as a backup plan for "customized sandbox images."

xingyaoww · 2024-07-21T17:45:11Z

the agenthub folder isn't copied into the sandbox for dev updates?

Yep, because agents suppose to run outside the sandbox, so probably there's little meaning in copying the code.

tobitege · 2024-07-21T17:55:34Z

the agenthub folder isn't copied into the sandbox for dev updates?

Yep, because agents suppose to run outside the sandbox, so probably there's little meaning in copying the code.

Hmm... are you sure? 😂 I don't see how the agentskills, ipython calls and browser stuff will work solely outside of the sandbox and only send commands... but it's getting late and I shouldn't get myself confused in the evening. 😁

tobitege

I think this is not yet quite ready.

tobitege · 2024-07-21T17:58:32Z

opendevin/runtime/client/runtime.py

@@ -81,14 +81,15 @@ async def ainit(self, env_vars: dict[str, str] | None = None):
            # NOTE: You can need set DEBUG=true to update the source code
            # inside the container. This is useful when you want to test/debug the
            # latest code in the runtime docker container.
-            update_source_code=config.debug,
+            update_source_code=self.sandbox_config.update_source_code,


This will be helpful! 👍

tobitege · 2024-07-21T18:04:49Z

opendevin/runtime/utils/runtime_build.py

-        '/opendevin/miniforge3/bin/mamba run -n base poetry install\n'
-        # for browser (update if needed)
-        'RUN apt-get update && cd /opendevin/code && /opendevin/miniforge3/bin/mamba run -n base poetry run playwright install --with-deps chromium\n'
+        '/opendevin/miniforge3/bin/mamba run -n base poetry install --no-interaction --no-root\n'


Should add --only main here.

There is no pyproject.toml file in the sandbox with this version of the method.

I think we do have pyproject.toml inside the sandbox

hmm... how did those get there, the "tar" doesn't copy them over, does it?

Yes, the .tar does copy that over as part of the python source build

And i added this to include the .lock as well:
https://github.com/OpenDevin/OpenDevin/blob/f3c23e80395bbe95fb8935ff1f8b2b6e563e3a5f/pyproject.toml#L9

Yes, the .tar does copy that over as part of the python source build

Ok, that I must have missed. Can you point me to where exactly that happens or do you mean "make build" here? 🤔

Yep - it is here:
https://github.com/OpenDevin/OpenDevin/blob/f3c23e80395bbe95fb8935ff1f8b2b6e563e3a5f/opendevin/runtime/utils/runtime_build.py#L22

We basically create a Python source distribution: https://docs.python.org/3.10/distutils/sourcedist.html

tobitege · 2024-07-21T18:06:05Z

opendevin/runtime/utils/runtime_build.py

+            LIBGL_MESA = 'libgl1'
+        else:
+            LIBGL_MESA = 'libgl1-mesa-glx'
+


Would add this at the top:

dockerfile_content = ( 'ENV DEBIAN_FRONTEND=noninteractive ' 'POETRY_NO_INTERACTION=1 ' 'POETRY_VIRTUALENVS_IN_PROJECT=1 ' 'POETRY_VIRTUALENVS_CREATE=1 ' 'POETRY_CACHE_DIR=/tmp/poetry_cache\n' )

(and "+=" in the below line then)

Is there any particular reason why we should add this when not adding it still works? I'm trying to see if we can make things as simple as possible..

They're also in the app dockerfile, thought they'd be needed:
https://github.com/OpenDevin/OpenDevin/blob/f3c23e80395bbe95fb8935ff1f8b2b6e563e3a5f/containers/app/Dockerfile#L18-L21

🤔 https://python-poetry.org/docs/configuration/#virtualenvsin-project

If not set explicitly, poetry by default will create a virtual environment under {cache-dir}/virtualenvs or use the {project-dir}/.venv directory if one already exists.

Now i see why the previous runs fails, because after rm -rf /root/.cache/pypoetry the virtual env will be removed.

I don't think we need this for the sandbox, because we will rm -rf /opendevin/code and copy a new version of the codebase there, if we enable POETRY_VIRTUALENVS_IN_PROJECT, it will delete the virtual env with rm -rf /opendevin/code , and we will have to re-install everything from scratch whenever we update the source code..

And POETRY_VIRTUALENVS_CREATE also default to true, so we probably don't need to set it either. https://python-poetry.org/docs/configuration/#virtualenvscreate

xingyaoww · 2024-07-21T18:34:54Z

Thanks @tobitege !

tobitege · 2024-07-21T18:35:36Z

Thanks @tobitege !

Thanks to you as well, happy to help! 🤗

tobitege and others added 10 commits July 19, 2024 23:41

test_runtime_client.py to test _execute_bash()

548bcb7

Merge branch 'main' into pexpect

611958e

Merge branch 'main' into pexpect

dfa43bb

runtime_build and runtime tweaks

14b808a

Merge branch 'main' into pexpect

573208e

fix in docker script

b1acc22

revert bash changes

a34e461

use sandbox_config.update_source_code to control source code update

9494ecd

add od_version to the sandbox tag

8c801b6

add doc instruction for update source code

219490a

yufansong reviewed Jul 21, 2024

View reviewed changes

xingyaoww added 3 commits July 21, 2024 12:24

do not remove whole poetry folder;

0cdac5a

add mamba clean

add missing newlines

1830ff4

Merge branch 'main' into xw/shrink-runtime-img-size

abbff99

xingyaoww marked this pull request as ready for review July 21, 2024 17:23

tobitege requested changes Jul 21, 2024

View reviewed changes

xingyaoww mentioned this pull request Jul 21, 2024

[Arch] Add runtime image build CI & clean up runtime build using jinja2 template #3055

Merged

tobitege approved these changes Jul 21, 2024

View reviewed changes

xingyaoww merged commit ce8a11a into main Jul 21, 2024
2 checks passed

xingyaoww deleted the xw/shrink-runtime-img-size branch July 21, 2024 18:34

tobitege mentioned this pull request Jul 22, 2024

(fix) image creation fixes for runtime_build.py #3056

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Arch] Shrink runtime image size #3051

[Arch] Shrink runtime image size #3051

xingyaoww commented Jul 20, 2024

yufansong Jul 21, 2024

tobitege Jul 21, 2024

tobitege commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

tobitege commented Jul 21, 2024 •

edited

Loading

tobitege commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

tobitege left a comment

tobitege Jul 21, 2024

tobitege Jul 21, 2024

tobitege Jul 21, 2024

xingyaoww Jul 21, 2024

tobitege Jul 21, 2024

xingyaoww Jul 21, 2024

xingyaoww Jul 21, 2024

tobitege Jul 21, 2024

xingyaoww Jul 21, 2024

tobitege Jul 21, 2024

xingyaoww Jul 21, 2024

tobitege Jul 21, 2024 •

edited

Loading

xingyaoww Jul 21, 2024

xingyaoww Jul 21, 2024 •

edited

Loading

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

[Arch] Shrink runtime image size #3051

[Arch] Shrink runtime image size #3051

Conversation

xingyaoww commented Jul 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tobitege commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

tobitege commented Jul 21, 2024 • edited Loading

tobitege commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

tobitege left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tobitege Jul 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xingyaoww Jul 21, 2024 • edited Loading

Choose a reason for hiding this comment

xingyaoww commented Jul 21, 2024

tobitege commented Jul 21, 2024

tobitege commented Jul 21, 2024 •

edited

Loading

tobitege Jul 21, 2024 •

edited

Loading

xingyaoww Jul 21, 2024 •

edited

Loading