Skip to content

[chore] cache docker images #38467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 10, 2025
Merged

Conversation

atoulme
Copy link
Contributor

@atoulme atoulme commented Mar 7, 2025

Try introducing a docker cache so we avoid hitting limits on Docker Hub.

Example:

https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13730940591/job/38407799262?pr=38466

=== FAIL: . TestObserverEmitsEndpointsIntegration (0.39s)
XXX: too many requests: &{AuthConfigs:map[[https://index.docker.io/v1/:{Username:](https://index.docker.io/v1/:%7BUsername:) Password: Auth:Z2l0aHViYWN0aW9uczozZDY0NzJiOS0zZDQ5LTRkMTctOWZjOS05MGQyNDI1ODA0M2I= Email: ServerAddress: IdentityToken: RegistryToken:}] HTTPHeaders:map[] PsFormat: ImagesFormat: NetworksFormat: PluginsFormat: VolumesFormat: StatsFormat: DetachKeys: CredentialsStore: CredentialHelpers:map[] Filename: ServiceInspectFormat: ServicesFormat: TasksFormat: SecretFormat: ConfigFormat: NodesFormat: PruneFilters:[] Proxies:map[] Experimental: StackOrchestrator: Kubernetes:<nil> CurrentContext: CLIPluginsExtraDirs:[] Aliases:map[]}    integration_test.go:40: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/extension/observer/dockerobserver/integration_test.go:40
        	Error:      	Received unexpected error:
        	            	create container: Error response from daemon: toomanyrequests: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit
        	Test:       	TestObserverEmitsEndpointsIntegration

@atoulme atoulme requested a review from a team as a code owner March 7, 2025 23:38
@atoulme atoulme requested a review from jpkrohling March 7, 2025 23:38
@atoulme
Copy link
Contributor Author

atoulme commented Mar 7, 2025

This works:
https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13731289975/job/38408718503?pr=38467

Post job cleanup.
Listing Docker images.
docker image list --format '{{ if ne .Repository "<none>" }}{{ .Repository }}{{ if ne .Tag "<none>" }}:{{ .Tag }}{{ end }}{{ else }}{{ .ID }}{{ end }}'
postgres:14
testcontainers/ryuk:0.11.0
nginx:1.17
Images present before restore step will be skipped; only new images will be saved.
docker save --output ~/.docker-images.tar postgres:14 testcontainers/ryuk:0.11.0 nginx:1.17
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib --files-from manifest.txt --use-compress-program zstdmt
Cache Size: ~175 MB (18[3](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13731289975/job/38408718503?pr=38467#step:9:3)167444 B)
Cache saved successfully

@atoulme
Copy link
Contributor Author

atoulme commented Mar 7, 2025

On subsequent builds, I see the cache being picked up. Here is a rerun of the same build:
https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13731289975/job/38409134055?pr=38467

Cache restored successfully
docker load --input ~/.docker-images.tar
Loaded image: postgres:14
Loaded image: testcontainers/ryuk:0.11.0
Loaded image: nginx:1.17

@songy23 songy23 added the ci-cd CI, CD, testing, build issues label Mar 10, 2025
@atoulme atoulme merged commit 20f3600 into open-telemetry:main Mar 10, 2025
199 checks passed
@github-actions github-actions bot added this to the next release milestone Mar 10, 2025
@atoulme atoulme deleted the cache_docker_images branch March 11, 2025 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cd CI, CD, testing, build issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants