compose: add support for referencing other services in additional_contexts

[emersion]

Description

See the last paragraph in https://docs.docker.com/reference/compose-file/build/#additional_contexts

This got added in compose-spec/compose-go#729 and docker-compose, but buildx is missing support.

[tonistiigi]

@crazy-max Can you check this? This should be supported.

[crazy-max]

We support additional contexts since Buildx v0.11.0:

buildx/bake/compose.go

Lines 92 to 96 in e0c67bf

	var additionalContexts map[string]string
	if s.Build.AdditionalContexts != nil {
	additionalContexts = map[string]string{}
	maps.Copy(additionalContexts, s.Build.AdditionalContexts)
	}

See the last paragraph in https://docs.docker.com/reference/compose-file/build/#additional_contexts

I guess you mean this part:

I tried to use this compose example from the docs:

services:
 base:
  build:
    context: .
    dockerfile_inline: |
      FROM alpine
      RUN echo foo
 my-service:
  build:
    context: .
    dockerfile_inline: |
      FROM base
      RUN echo bar
    additional_contexts:
      base: service:base

But it fails because it exports base as docker.io/library/3074-base instead of docker.io/library/base. Seems it uses my working dir (or project dir) as prefix (cc @glours @ndeloof):

$ docker compose build
INFO[0000] additional_context with "service:"" is better supported when delegating build go bake. Set COMPOSE_BAKE=true 
#0 building with "default" instance using docker driver

#1 [base internal] load build definition from Dockerfile
#1 transferring dockerfile: 62B done
#1 DONE 0.0s

#2 [base internal] load metadata for docker.io/library/alpine:latest
#2 DONE 0.0s

#3 [base internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [base 1/2] FROM docker.io/library/alpine:latest
#4 DONE 0.0s

#5 [base 2/2] RUN echo foo
#5 CACHED

#6 [base] exporting to image
#6 exporting layers done
#6 writing image sha256:1abd84a2c253948a7b5d198793e42c1384c403a394c3934b2dcbbf15ba8119fb done
#6 naming to docker.io/library/3074-base done
#6 DONE 0.0s

#7 [base] resolving provenance for metadata file
#7 DONE 0.0s

#8 [my-service internal] load build definition from Dockerfile
#8 transferring dockerfile: 60B done
#8 DONE 0.0s

#9 [my-service internal] load metadata for docker.io/library/base:latest
#9 ...

#10 [my-service auth] library/base:pull token for registry-1.docker.io
#10 DONE 0.0s

#9 [my-service internal] load metadata for docker.io/library/base:latest
#9 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [my-service internal] load metadata for docker.io/library/base:latest:
------
failed to solve: base: failed to resolve source metadata for docker.io/library/base:latest: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

$ docker compose version
Docker Compose version v2.33.1-desktop.1

And looking at the first info message:

INFO[0000] additional_context with "service:"" is better supported when delegating build go bake. Set COMPOSE_BAKE=true

Delegating to bake should work.

[crazy-max]

With changes from compose-spec/compose-go#729 I think we need to adapt service: to target: so it would work with bake:

$ docker buildx bake
#0 building with "default" instance using docker driver

#1 [internal] load local bake definitions
#1 reading compose.yml 250B / 250B done
#1 DONE 0.0s

#2 [base internal] load build definition from Dockerfile
#2 transferring dockerfile: 62B done
#2 DONE 0.0s

#3 [base internal] load metadata for docker.io/library/alpine:latest
#3 DONE 0.0s

#4 [base internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [my-service internal] load build definition from Dockerfile
#5 transferring dockerfile: 60B done
#5 DONE 0.0s

#4 [my-service internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#6 [base 1/2] FROM docker.io/library/alpine:latest
#6 DONE 0.0s

#7 [base 2/2] RUN echo foo
#7 CACHED

#8 [base] exporting to image
#8 exporting layers done
#8 writing image sha256:edb2dd0589b0c666ed65039655f4e629c5df253f00d3ebc71857fdd175c6f816 done
#8 DONE 0.0s

#9 [my-service 1/2] RUN echo bar
#9 0.318 bar
#9 DONE 0.4s

#10 [my-service] exporting to image
#10 exporting layers 0.0s done
#10 writing image sha256:1026df0ef881d4343514ff9fc61f63e6f25953891d4163f9139e4c2a3d57159c done
#10 DONE 0.1s

View build details:
  base: docker-desktop://dashboard/build/default/default/tpmv62skb2rihunebxsraacmu
  my-service: docker-desktop://dashboard/build/default/default/lrj0i5hg6618tpaypz9fuybhh

[ndeloof]

see https://github.com/docker/compose/blob/main/pkg/compose/build.go#L580-L586

[ndeloof]

.. and https://github.com/docker/compose/blob/main/pkg/compose/build_bake.go#L337-L338 for bake integration

[tonistiigi]

see https://github.com/docker/compose/blob/main/pkg/compose/build.go#L580-L586

That would not work. The design of linked targets in bake is that dependencies can be linked lazily and then all the targets built together in parallel. Local images names are only for Docker driver. If you replace service: with image name then both targets will start to build in parallel, with the image either being loaded from registry(as we see from @crazy-max logs) or for Docker driver if old image exists from previous build then that would be used(unless image resolve mode to pull).

[ndeloof]

@tonistiigi this one is for direct buildkit execution, bake integration uses target: https://github.com/docker/compose/blob/main/pkg/compose/build_bake.go#L337-L338

[emersion]

Many thanks to all!