From d31505ff365b9b053b4b602f4d5480b81c71b1ef Mon Sep 17 00:00:00 2001 From: Rich Lander Date: Wed, 10 Aug 2022 16:31:14 -0700 Subject: [PATCH] Add proposal on non-root container images --- accepted/2022/non-root-container-images.md | 162 +++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 accepted/2022/non-root-container-images.md diff --git a/accepted/2022/non-root-container-images.md b/accepted/2022/non-root-container-images.md new file mode 100644 index 000000000..18bed4ee8 --- /dev/null +++ b/accepted/2022/non-root-container-images.md @@ -0,0 +1,162 @@ +# Proposal for .NET Non-Root Container Images + +We all know that [running as root is considered harmful](https://unix.stackexchange.com/questions/1052/concern-about-logging-in-as-root-overrated) yet it is commonplace to [run container images that way](https://www.redhat.com/blog/understanding-root-inside-and-outside-container). We can make it easier to run .NET container images as non-root. + +## Goals + +The following will improve security for users: + +- Enable all .NET images to be run as a non-root user. +- Exposed ports (via `ASPNETCORE_URLS`) are constant across all image types. +- Images do not use privileged resources by default. +- It is easy to use any combination of root and non-root images in a deployment. + +## Plan + +We can satisfy those goals (not necessarily in that order) across multiple releases. + +- .NET 7 + - Add same non-root user in both rootful and non-root images + - Expose same non-root ports -- `8080` and `8443` (for HTTPS) -- in both rootful and non-root images + - Continue to expose port `80` in rootful images, for compatibility. + - Transition `mcr.microsoft.com/dotnet/samples` to non-root images (which by definition means not exposing port `80`). +- .NET 8 + - Remove port `80` from rootful images. + - Announce that all .NET images are turnkey non-root capable. + +Notes: + +- Actual port numbers are TBD. +- Some environments [allow access to privileged ports to non-root users]( +https://github.com/dotnet/dotnet-docker/issues/3796). + +The rest of this document provides detailed technical context if that's helpful, but is otherwise unnecessary to read. + +## Privileged ports + +[Ports under `1024` are privileged (require `root` permission)](https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html), while ports `>= 1024` can be accessed by a regular user. + +Kestrel (ASP.NET Core web server) is configured to [listen on port `80`](https://github.com/dotnet/dotnet-docker/blob/7cf01d82858fcc3824574fb92580c4151954699a/src/runtime-deps/6.0/jammy/amd64/Dockerfile#L19) in .NET team provided containers. As a result, we have a significant `root` dependency that we need to address. + +In contrast, Red Hat [configures images in OpenShift with `8080` and `8443`](https://github.com/dotnet/aspnetcore/issues/43149#issuecomment-1209525031). That's quite self-descriptive and works great for `non-root` scenarios. + +It turns out we already use port `8080` for our [non-root Mariner images](https://github.com/dotnet/dotnet-docker/blob/7cf01d82858fcc3824574fb92580c4151954699a/src/runtime-deps/6.0/cbl-mariner2.0-distroless/amd64/Dockerfile#L52-L53). + +Note: Mariner images are public, but only supported for Microsoft internal usage. + +For .NET 7, we should publish images the following way (replacing the existing `ASPNETCORE_URLS` definition): + +- Non-root images: `ASPNETCORE_URLS=http://+:8080;https://+:8443` +- Rootful images: `ASPNETCORE_URLS=http://+:80;http://+:8080;https://+:8443` + +That approach will enable users to move easily between root and non-root images provided that they adopt the `8xxx` ports. It will also enable TLS usage without needing to re-specify this `ENV`. + +At a later point -- hopefully .NET 8 -- we should remove port `80` from the rootful images. The document demonstrates why removing port `80` enables us to significantly improve adoption of non-root containers by our "user" users. + +Alternatively, we could adopt ports `5000` and `5001`. ASP.NET Core already uses those as the defaults for development. It's a tradeoff between ports that are more self-descriptive and ports that are more idiomatic for ASP.NET Core. + +## .NET Container Hosting Infrastructure + +There may infrastructure that exclusively hosts or tests .NET containers and that is hard-coded to port `80`. If those services only support rootful container images, that may be OK. However, it's still not a great model. Instead, infrastructure should configure everything about port publishing and exposure itself to ensure reliability. + +For example, imagine a service that was solely using the [`aspnetapp` sample](https://mcr.microsoft.com/product/dotnet/samples/about). We may transition that container image to non-root purely as a technology demonstration. Anyone relying on Kestrel listening on port `80` within the image and by extension being rootful would be broken. + +Instead, such a service should redefine the `ASPNETCORE_URLS` ENV to a known non-privileged port of their choosing, like in the following example. + +```bash +% docker run --rm -p 8080:8080 -d -e ASPNETCORE_URLS=http://+:8080 mcr.microsoft.com/dotnet/samples:aspnetapp +5130b894c5a2b8c7bb59e2583eb6efe8adb21e8908d97a75e4fae01fce8a038e +rich@MacBook-Air-2 aspnetapp % curl http://localhost:8080/Environment +{"runtimeVersion":".NET 6.0.6","osVersion":"Linux 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022","osArchitecture":"Arm64","processorCount":4,"totalAvailableMemoryBytes":4108652544,"memoryLimit":0,"memoryUsage":0}% +``` + +The matching ports guarantees that the app will work. Note that the `8080:8080` host:guest port mapping don't need to match on both side. It's the the guest and `ENV` ports that need to match. + +The following works equally well. + +```bash +rich@MacBook-Air-2 aspnetapp % docker run --rm -p 8088:8080 -d -e ASPNETCORE_URLS=http://+:8080 mcr.microsoft.com/dotnet/samples:aspnetapp +44bbd26fa870cfa654f0652d188ea2ac7015231d2e86c27b6747a30bd73c2f89 +rich@MacBook-Air-2 aspnetapp % curl http://localhost:8088/Environment +{"runtimeVersion":".NET 6.0.6","osVersion":"Linux 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022","osArchitecture":"Arm64","processorCount":4,"totalAvailableMemoryBytes":4108652544,"memoryLimit":0,"memoryUsage":0}% +``` + +## Running images as non-root + +You can run .NET images as non-root today, but it isn't straightforward. We can improve that. + +There are a spectrum of images we could publish: + +- Root +- Root + optional non-root user +- non-root + +We almost exclusively publish rootful images today. We may publish non-root images in the future. In the meantime, we can consider publishing root images with an optional non-root user. That would make it very easy to run .NET images as non-root. + +The `docker` CLI enables specifying a user with `docker run`. Let's try that. + +```bash +$ docker run --rm -it -p 8088:80 -u app mcr.microsoft.com/dotnet/samples:aspnetapp +docker: Error response from daemon: unable to find user app: no matching entries in passwd file. +``` + +That's not useful and makes sense. We cannot use a user that hasn't been added to the container image. + +We can [add one](https://gist.github.com/richlander/34e514446afece01252f19c2a18c3222), however. + +```dockerfile +FROM mcr.microsoft.com/dotnet/samples:aspnetapp + +RUN groupadd \ + --system \ + --gid=101 \ + app \ + && adduser \ + --uid 101 \ + --gid 101 \ + --shell /bin/false \ + --no-create-home \ + --system \ + app +``` + +Now, lets try that, both with and w/o a user specified. + +```bash +$ docker run --rm -it -p 8088:80 aspnetappwithuser + + Now listening on: http://[::]:80 +``` + +That works as per normal. + +Now, let's run as our user. + +```bash +$ docker run --rm -it -p 8088:80 -u app aspnetappwithuser + +Unhandled exception. System.Net.Sockets.SocketException (13): Permission denied +``` + +Excellent. It fails. Now let's re-configure the port. + +```bash +$ docker run --rm -it -p 8088:8080 -u app -e ASPNETCORE_URLS=http://+:8080 aspnetappwithuser + + Now listening on: http://[::]:8080 +``` + +Works like a dream. + +We can also ask the container which account it is working under. + +```bash +rich@kamloops:~/aspnetapp$ docker run --rm -d -p 8088:8080 -u app -e ASPNETCORE_URLS=http://+:8080 aspnetappwithuser +cf0f115399059600fd933697dcb64cbf255e459b8a0e2d2b44b53aa71f2029de +rich@kamloops:~/aspnetapp$ docker exec cf0f115399059600fd933697dcb64cbf255e459b8a0e2d2b44b53aa71f2029de whoami +app +``` + +This little scenario also demonstrates why removing port `80` from the rootful images would be useful. We wouldn't have to redefine `ASPNETCORE_URLS` in order to switch between `root` and `app` for images that are rootful by default. That would be very nice. + +For clarity, the approach that was used to add the user to the `aspnetapp` image was just a proof-of-concept. The intent is to add this user in the `runtime-deps` images (or `runtime` for Windows).