Separate needs for ot_agent_standalone builds by architecture #36769
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
short review
Gitlab CI Configuration ChangesModified Jobs.docker_build_ot_agent_standalone .docker_build_ot_agent_standalone:
- needs:
- - job: docker_build_base_image_arm64
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
variables:
BUILD_ARG: --target release
BUILD_CONTEXT: Dockerfiles/otel-agent
FLATTEN_IMAGE: false
IMAGE: registry.ddbuild.io/ci/datadog-agent/otel-agent
TAG_SUFFIX: -7docker_build_ot_agent_standalone_amd64 docker_build_ot_agent_standalone_amd64:
image: registry.ddbuild.io/ci/datadog-agent-buildimages/docker_x64$DATADOG_AGENT_BUILDIMAGES_SUFFIX:$DATADOG_AGENT_BUILDIMAGES
needs:
- - job: docker_build_base_image_arm64
? -
+ - job: docker_build_base_image_amd64
? +
retry: 2
rules:
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- when: on_success
script:
- TAG_SUFFIX=${TAG_SUFFIX:-}
- BUILD_ARG=${BUILD_ARG:-}
- "if [[ \"$BUCKET_BRANCH\" == \"nightly\" && ( \"$IMAGE\" =~ \"ci/datadog-agent/agent\"\
\ || \"$IMAGE\" =~ \"ci/datadog-agent/cluster-agent\" || \"$IMAGE\" =~ \"ci/datadog-agent/cws-instrumentation\"\
\ || \"$IMAGE\" =~ \"ci/datadog-agent/otel-agent\" ) ]]; then\n export ECR_RELEASE_SUFFIX=\"\
-nightly\"\nelse\n export ECR_RELEASE_SUFFIX=${CI_COMMIT_TAG+-release}\nfi\n"
- AGENT_BASE_IMAGE_TAG=registry.ddbuild.io/ci/datadog-agent/agent-base-image${ECR_RELEASE_SUFFIX}:v${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}-$ARCH
- TARGET_TAG=${IMAGE}${ECR_RELEASE_SUFFIX}:v${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}$TAG_SUFFIX-$ARCH
- DOCKER_LOGIN=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DOCKER_REGISTRY_RO user)
|| exit $?
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $DOCKER_REGISTRY_RO token | docker login
--username "$DOCKER_LOGIN" --password-stdin "$DOCKER_REGISTRY_URL"
- EXIT="${PIPESTATUS[0]}"; if [ $EXIT -ne 0 ]; then echo "Unable to locate credentials
needs gitlab runner restart"; exit $EXIT; fi
- docker buildx build --no-cache --push --pull --platform linux/$ARCH --build-arg
AGENT_BASE_IMAGE_TAG=${AGENT_BASE_IMAGE_TAG} --build-arg CIBUILD=true --build-arg
GENERAL_ARTIFACTS_CACHE_BUCKET_URL=${GENERAL_ARTIFACTS_CACHE_BUCKET_URL} $BUILD_ARG
--build-arg DD_GIT_REPOSITORY_URL=https://github.com/DataDog/datadog-agent --build-arg
DD_GIT_COMMIT_SHA=${CI_COMMIT_SHA} --file $BUILD_CONTEXT/Dockerfile --tag ${TARGET_TAG}
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" --label
"org.opencontainers.image.authors=Datadog <[email protected]>" --label "org.opencontainers.image.source=https://github.com/DataDog/datadog-agent"
--label "org.opencontainers.image.version=$(dda inv agent.version)" --label "org.opencontainers.image.revision=${CI_COMMIT_SHA}"
--label "org.opencontainers.image.vendor=Datadog, Inc." --label "target=none"
$BUILD_CONTEXT
- FLATTEN_IMAGE=${FLATTEN_IMAGE:-true}
- "if [[ \"$FLATTEN_IMAGE\" == \"true\" ]]; then\n crane flatten -t ${TARGET_TAG}\
\ ${TARGET_TAG}\nfi\n"
stage: container_build
tags:
- arch:amd64
timeout: 30m
variables:
ARCH: amd64
BUILD_ARG: --target release
BUILD_CONTEXT: Dockerfiles/otel-agent
FLATTEN_IMAGE: false
IMAGE: registry.ddbuild.io/ci/datadog-agent/otel-agent
TAG_SUFFIX: -7Changes Summary
ℹ️ Diff available in the job log. |
alopezz
added
changelog/no-changelog
qa/no-code-change
Uncompressed package size comparisonComparison with ancestor Diff per package
Decision✅ Passed |
Static quality checks✅ Please find below the results from static quality gates Successful checksInfo
|
spencergilbert
approved these changes
May 8, 2025
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
dd-mergequeue
bot
deleted the
alopez/fix_build_ot_agent_standalone_build_deps
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Specifies the actual dependency for each architecture of the ot_agent_standalone builds.
Motivation
Fixing this kind of error: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/928465297.
Both
docker_build_ot_agent_standalone_amd64anddocker_build_ot_agent_standalone_arm64are depending ondocker_build_base_image_arm64on GitLab terms, but theamd64image actually depends on its matching base image being present in the registry. This leads to situations where the amd64 build can fail because its base image is still not there. In the above example, the base image build failed for amd64 on the first attempt while the arm64 didn't, causing it to fail.Describe how you validated your changes
Pipeline passing would be enough.
Possible Drawbacks / Trade-offs
Additional Notes