Skip to content

Upgrade the docker version to 23.0.6 #5436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ansible/group_vars/all
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ invoker:
userMemory: "{{ invoker_user_memory | default('2048m') }}"
# Specify if it is allowed to deploy more than 1 invoker on a single machine.
allowMultipleInstances: "{{ invoker_allow_multiple_instances | default(false) }}"
# Specify if it should use docker-runc or docker to pause/unpause containers
# Specify if it should use runc or docker to pause/unpause containers
useRunc: "{{ invoker_use_runc | default(true) }}"
docker:
become: "{{ invoker_docker_become | default(false) }}"
Expand Down
4 changes: 2 additions & 2 deletions ansible/roles/invoker/tasks/clean.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,9 @@
- name: pause/resume at runc-level to restore docker consistency
shell: |
DOCKER_PAUSED=$(docker ps --filter status=paused --filter name=wsk{{ invoker_index }} -q --no-trunc)
for C in $DOCKER_PAUSED; do docker-runc --root {{ invoker.docker.runcdir }} pause $C; done
for C in $DOCKER_PAUSED; do runc --root {{ invoker.docker.runcdir }} pause $C; done
DOCKER_RUNNING=$(docker ps --filter status=running --filter name=wsk{{ invoker_index }} -q --no-trunc)
for C2 in $DOCKER_RUNNING; do docker-runc --root {{ invoker.docker.runcdir }} resume $C2; done
for C2 in $DOCKER_RUNNING; do runc --root {{ invoker.docker.runcdir }} resume $C2; done
TOTAL=$(($(echo $DOCKER_PAUSED | wc -w)+$(echo $DOCKER_RUNNING | wc -w)))
echo "Handled $TOTAL remaining actions."
register: runc_output
Expand Down
4 changes: 2 additions & 2 deletions ansible/roles/invoker/tasks/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -382,8 +382,8 @@
{{ docker_sock | default('/var/run/docker.sock') }}:/var/run/docker.sock"
###
# The root runc directory varies based on the version of docker and runc.
# When docker>=18.06 uses docker-runc the directory is /run/docker/runtime-runc/moby.
# While docker-runc itself uses /run/runc for a root user or /run/user/<uid>/runc for a non-root user.
# When docker>=18.06 uses runc the directory is /run/docker/runtime-runc/moby.
# While runc itself uses /run/runc for a root user or /run/user/<uid>/runc for a non-root user.
# Currently, the invoker is running as a root user so the below configuration works as expected.
# But when the invoker needs to run as a non-root user or the version docker needs to be changed,
# the following configuration should be properly updated as well.
Expand Down
6 changes: 3 additions & 3 deletions core/controller/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,18 @@ ENV SWAGGER_UI_DOWNLOAD_SHA256=3d7ef5ddc59e10f132fe99771498f0f1ba7a2cbfb9585f986
# If you change the docker version here, it has implications on invoker runc support.
# Docker server version and the invoker docker version must be the same to enable runc usage.
# If this cannot be guaranteed, set `invoker_use_runc: false` in the ansible env.
ENV DOCKER_VERSION=18.06.3-ce
ENV DOCKER_VERSION=23.0.6

RUN apk add --update openssl

# Uncomment to fetch latest version of docker instead: RUN wget -qO- https://get.docker.com | sh
# Install docker client
RUN curl -sSL -o docker-${DOCKER_VERSION}.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker-runc && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/runc && \
rm -f docker-${DOCKER_VERSION}.tgz && \
chmod +x /usr/bin/docker && \
chmod +x /usr/bin/docker-runc
chmod +x /usr/bin/runc
##################################################################################################

# Install swagger-ui
Expand Down
6 changes: 3 additions & 3 deletions core/controller/Dockerfile-debian
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,18 @@ ENV SWAGGER_UI_DOWNLOAD_SHA256=3d7ef5ddc59e10f132fe99771498f0f1ba7a2cbfb9585f986
# If you change the docker version here, it has implications on invoker runc support.
# Docker server version and the invoker docker version must be the same to enable runc usage.
# If this cannot be guaranteed, set `invoker_use_runc: false` in the ansible env.
ENV DOCKER_VERSION=18.06.3-ce
ENV DOCKER_VERSION=23.0.6

RUN apt-get -y install openssl

# Uncomment to fetch latest version of docker instead: RUN wget -qO- https://get.docker.com | sh
# Install docker client
RUN curl -sSL -o docker-${DOCKER_VERSION}.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker-runc && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/runc && \
rm -f docker-${DOCKER_VERSION}.tgz && \
chmod +x /usr/bin/docker && \
chmod +x /usr/bin/docker-runc
chmod +x /usr/bin/runc
##################################################################################################

# Install swagger-ui
Expand Down
6 changes: 3 additions & 3 deletions core/invoker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ FROM scala

ENV UID=1001 \
NOT_ROOT_USER=owuser \
DOCKER_VERSION=18.06.3-ce
DOCKER_VERSION=23.0.6
# If you change the docker version here, it has implications on invoker runc support.
# Docker server version and the invoker docker version must be the same to enable runc usage.
# If this cannot be guaranteed, set `invoker_use_runc: false` in the ansible env.
Expand All @@ -31,10 +31,10 @@ RUN apk add --update openssl
# Install docker client
RUN curl -sSL -o docker-${DOCKER_VERSION}.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker-runc && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/runc && \
rm -f docker-${DOCKER_VERSION}.tgz && \
chmod +x /usr/bin/docker && \
chmod +x /usr/bin/docker-runc
chmod +x /usr/bin/runc

ADD build/distributions/invoker.tar ./

Expand Down
6 changes: 3 additions & 3 deletions core/invoker/Dockerfile-debian
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ FROM scala

ENV UID=1001 \
NOT_ROOT_USER=owuser
ENV DOCKER_VERSION=18.06.3-ce
ENV DOCKER_VERSION=23.0.6
# If you change the docker version here, it has implications on invoker runc support.
# Docker server version and the invoker docker version must be the same to enable runc usage.
# If this cannot be guaranteed, set `invoker_use_runc: false` in the ansible env.
Expand All @@ -31,10 +31,10 @@ RUN apt-get -y install openssl
# Install docker client
RUN curl -sSL -o docker-${DOCKER_VERSION}.tgz https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker-runc && \
tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/runc && \
rm -f docker-${DOCKER_VERSION}.tgz && \
chmod +x /usr/bin/docker && \
chmod +x /usr/bin/docker-runc
chmod +x /usr/bin/runc

ADD build/distributions/invoker.tar ./

Expand Down
2 changes: 1 addition & 1 deletion core/invoker/src/main/resources/application.conf
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ whisk {
}

docker.container-factory {
# Use runc (docker-runc) for pause/resume functionality in DockerContainerFactory
# Use runc for pause/resume functionality in DockerContainerFactory
use-runc: true
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ object DockerContainer {
* @param network network to launch the container in
* @param dnsServers list of dns servers to use in the container
* @param name optional name for the container
* @param useRunc use docker-runc to pause/unpause container?
* @param useRunc use runc to pause/unpause container?
* @return a Future which either completes with a DockerContainer or one of two specific failures
*/
def create(transid: TransactionId,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ class RuncClient(timeouts: RuncClientTimeouts = loadConfigOrThrow[RuncClientTime

// Determines how to run docker. Failure to find a Docker binary implies
// a failure to initialize this instance of DockerClient.
protected val runcCmd: Seq[String] = Seq("/usr/bin/docker-runc")
protected val runcCmd: Seq[String] = Seq("/usr/bin/runc")

def pause(id: ContainerId)(implicit transid: TransactionId): Future[Unit] =
runCmd(Seq("pause", id.asString), timeouts.pause).map(_ => ())
Expand Down
8 changes: 4 additions & 4 deletions docs/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,12 @@ OpenWhisk used to support both shared state and a sharding model. The former has

The sharding loadbalancer has the caveat of being limited in its scalability in its current implementation. It uses "horizontal" sharding, which means that the slots on each invoker are evenly divided to the loadbalancers. For example: In a system with 2 loadbalancers and invokers which have 16 slots each, each loadbalancer would get 8 slots on each invoker. In this specific case, a cluster of loadbalancers > 16 instances does not make sense, since each loadbalancer would only have a fraction of a slot above that. The code guards against that but it is strongly recommended not to deploy more sharding loadbalancers than there are slots on each invoker.

# Invoker use of docker-runc
# Invoker use of runc

To improve performance, Invokers attempt to maintain warm containers for frequently executed actions. To optimize resource usage, the action containers are paused/unpaused between invocations. The system can be configured to use either docker-runc or docker to perform the pause/unpause operations by setting the value of the environment variable INVOKER_USE_RUNC to true or false respectively. If not set, it will default to true (use docker-runc).
To improve performance, Invokers attempt to maintain warm containers for frequently executed actions. To optimize resource usage, the action containers are paused/unpaused between invocations. The system can be configured to use either runc or docker to perform the pause/unpause operations by setting the value of the environment variable INVOKER_USE_RUNC to true or false respectively. If not set, it will default to true (use runc).

Using docker-runc obtains significantly better performance, but requires that the version of docker-runc within the invoker container is an exact version match to the docker-runc of the host environment. Failure to get an exact version match will results in error messages like:
Using runc obtains significantly better performance, but requires that the version of runc within the invoker container is an exact version match to the runc of the host environment. Failure to get an exact version match will results in error messages like:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still a requirement with runc that versions must be an exact match? Makes it very hard to use runc in serverless environment where you may control the invoker container but not the underlying server.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should not be an exact match, but some versions have breaking changes.
For example, the command's name is changed from docker-runc to runc.
The root volume directory was also different in the previous version.
https://github.com/apache/openwhisk/pull/4430/files#diff-072209721097df0ae37ac99015b28844fec1f9b390314782671445b4f80af622R185

I assume if there is no such breaking change, runc would be compatible, but anyway, it would be better to use the same version if possible.

```
2017-09-29T20:15:54.551Z] [ERROR] [#sid_102] [RuncClient] code: 1, stdout: , stderr: json: cannot unmarshal object into Go value of type []string [marker:invoker_runc.pause_error:6830148:259]
```
When a docker-runc operations results in an error, the container will be killed by the invoker. This results in missed opportunities for container reuse and poor performance. Setting INVOKER_USE_RUNC to false can be used as a workaround until proper usage of docker-runc can be configured for the deployment.
When a runc operations results in an error, the container will be killed by the invoker. This results in missed opportunities for container reuse and poor performance. Setting INVOKER_USE_RUNC to false can be used as a workaround until proper usage of runc can be configured for the deployment.
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ class RuncClientTests
implicit val transid = TransactionId.testing
val id = ContainerId("Id")

val runcCommand = "docker-runc"
val runcCommand = "runc"

/** Returns a RuncClient with a mocked result for 'executeProcess' */
def runcClient(result: Future[String]) = new RuncClient()(global) {
Expand Down