when can we expect distroless based on bullseye?

[sambercovici]

debian bullseye is expected to be GA on 14th August.
when can we expect to get distroless based on debian 11 (aka. bullseye)

[loosebazooka]

We're working on it, but need to update our builds to remove the dependency on snapshot.debian.org (which has been somewhat flaky lately). There's no actual timeline for this except "near future".

[sambercovici]

As I needed it now, I have an ugly hacked clone at: https://github.com/sambercovici/distroless

[loosebazooka]

I'm a little surprised nothing went wrong. There is no Packages.gz in the updates channel for bullseye. Only a Packages.xz. Code needs to be updated to handle that. Or it's just silently ignoring the updates channel all together.

https://snapshot.debian.org/archive/debian/20210806T092041Z/dists/bullseye-updates/main/binary-amd64/

[chanseokoh]

It's going to be all .xz going forward? #463

[joshuagl]

We're working on it, but need to update our builds to remove the dependency on snapshot.debian.org (which has been somewhat flaky lately). There's no actual timeline for this except "near future".

I'm surprised to read about a move away from snapshot.debian.org, is that documented somewhere? I've commented before that:

One of the nice, presumably intended, consequences of distroless using snapshots.debian.org, is that one can recreate a given distroless image even when the main Debian release archives have moved on.

If we switch away from snapshots.debian.org, then we lose the ability to reliably reproduce distroless container images.

That is because package versions which are no longer listed by a package index are reaped from the pool on the main package repository and its mirrors. Package versions stop being listed by the package index when a newer version replaces them.

[dlorenc]

I'm surprised to read about a move away from snapshot.debian.org, is that documented somewhere? I've commented before that:

Sorry we're keeping the snapshotting, just not relying on that service because it's been very very flaky. @loosebazooka setup a job to copy things from there over to a GCS bucket that we can use directly for builds over here: #783

[joshuagl]

Thanks for the fast reply Dan!

While I understand the logic, that seems a bit of a shame as it reduces the "transparency" of distroless images (harder to map to upstream Debian contents).

Presumably that also makes it harder to build custom distroless images with Bazel which only include the components the Bazel target needs (when those custom images require dependencies not in the gcs cache).

[dlorenc]

Yeah, it's really unfortunate. The "flakiness" has pretty much been a showstopper though, we've gone days/weeks without being able to get a single build to complete successfully.

@loosebazooka and @priyawadhwa are working on some longer term plans to actually build packages from source reproducibly and use those rather than the snapshot binaries directly (and checking the reproduction).

We'd actually like to use the distroless images as a showcase/dogfooding examlpe for SLSA over time, getting everything built from source following all the best practices available.

[joshuagl]

@loosebazooka and @priyawadhwa are working on some longer term plans to actually build packages from source reproducibly and use those rather than the snapshot binaries directly (and checking the reproduction).

This sounds like a good way forwards, I'm very interested in this!

We'd actually like to use the distroless images as a showcase/dogfooding examlpe for SLSA over time, getting everything built from source following all the best practices available.

This is a great idea. I was speculating about trying to identify the SLSA levels of artefacts produced by distributions as a starting point for this kind of effort. It would be great for this best practices work to feed into the upstream.

There has been some work with the Debian reproducibility folks to integrate in-toto and the Qubes OS folks are already using in-toto.

[loosebazooka]

While I understand the logic, that seems a bit of a shame as it reduces the "transparency" of distroless images (harder to map to upstream Debian contents).

This cache will still contain debian metadata + will contain the original full Packages.xz and signature file we sourced from (work in progress) so at least you can trace it back to debian.

[loosebazooka]

Presumably that also makes it harder to build custom distroless images with Bazel which only include the components the Bazel target needs (when those custom images require dependencies not in the gcs cache).

Using the cache for the base distroless build will not prevent other users from pointing to a debian repository to add on packages that they want.

[joshuagl]

At the risk of overloading this issue (please redirect me to a more appropriate place to discuss if one exists) I noticed DebConf 21 included a talk on "Making use of snapshot.debian.org for fun and profit" which includes mention of

a new testing implementation of snapshot service without throttling https://debian.notset.fr/snapshot

[evanj]

FWIW: I think this issue can be closed? The Distroless images have been updated to debian11.