Skip to content

Commit 80b08d1

Browse files
authored
Do not allow for any pushes to customer jars if the branch is dirty and not pushed to remote (#385)
## Summary I've been seeing that it's difficult to track what changes went into artifacts we push to etsy and canary. Especially when it comes to tracking performance regressions for spark jobs one day to the next. Adding a check to not allow any pushes to any customer artifacts if the branch is dirty. All changes need to at least be pushed to remote. And adding a metadata tag of the commit and branch ## Checklist - [ ] Added Unit Tests - [ ] Covered by existing CI - [ ] Integration tested - [ ] Documentation update <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced consistency checks during the build and upload process to verify that local changes are committed and branches are in sync. - Enhanced artifact metadata now includes additional context about the code state at the time of upload. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent deafaeb commit 80b08d1

File tree

1 file changed

+24
-5
lines changed

1 file changed

+24
-5
lines changed

distribution/build_and_upload_gcp_artifacts.sh

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,24 @@
11
#!/bin/bash
22

3+
4+
if [[ -n $(git diff HEAD) ]]; then
5+
echo "Error: You have uncommitted changes. Please commit and push them to git so we can track them."
6+
exit 1
7+
fi
8+
9+
# Get current branch name
10+
local_branch=$(git rev-parse --abbrev-ref HEAD)
11+
12+
# Fetch latest from remote
13+
git fetch origin $local_branch
14+
15+
# Check if local is behind remote
16+
if [[ -n $(git diff HEAD..origin/$local_branch) ]]; then
17+
echo "Error: Your branch is not in sync with remote"
18+
echo "Please push your local changes and sync your local branch $local_branch with remote"
19+
exit 1
20+
fi
21+
322
set -e
423

524
SCRIPT_DIRECTORY=$(dirname -- "$(realpath -- "$0")")
@@ -85,11 +104,11 @@ function upload_to_gcp() {
85104
for element in "${customer_ids_to_upload[@]}"
86105
do
87106
ELEMENT_JAR_PATH=gs://zipline-artifacts-$element/jars
88-
gcloud storage cp "$CLOUD_GCP_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date)"
89-
gcloud storage cp "$CLOUD_GCP_SUBMITTER_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date)"
90-
gcloud storage cp "$SERVICE_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date)"
91-
gcloud storage cp "$EXPECTED_ZIPLINE_WHEEL" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date)"
92-
gcloud storage cp "$FLINK_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date)"
107+
gcloud storage cp "$CLOUD_GCP_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
108+
gcloud storage cp "$CLOUD_GCP_SUBMITTER_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
109+
gcloud storage cp "$SERVICE_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
110+
gcloud storage cp "$EXPECTED_ZIPLINE_WHEEL" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
111+
gcloud storage cp "$FLINK_JAR" "$ELEMENT_JAR_PATH" --custom-metadata="zipline_user=$USER,updated_date=$(date),commit=$(git rev-parse HEAD),branch=$(git rev-parse --abbrev-ref HEAD)"
93112
done
94113
echo "Succeeded"
95114
break;;

0 commit comments

Comments
 (0)