Skip to content

[Github Actions] Rename docker secrets and parameterize docker user #13297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 28, 2022
Merged

[Github Actions] Rename docker secrets and parameterize docker user #13297

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9383c9c
Rename docker secrets and parameterize docker user
evantahler May 28, 2022
8ad0459
Apply suggestions from code review
evantahler May 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/build-and-push-branch/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ runs:
- name: Login to Docker (on Master)
uses: docker/login-action@v1
with:
username: airbytebot
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ inputs.dockerhub_token }}

- name: Push Docker Images
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/build-connector-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,8 @@ jobs:
id: build
env:
PR_NUMBER: ${{ github.event.number }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
# Oracle expects this variable to be set. Although usually present, this is not set by default on Github virtual runners.
TZ: UTC
# - name: Test ${{ github.event.inputs.connector }}
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/gke-kube-test-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,8 @@ jobs:
env:
USER: root
HOME: /home/runner
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
ACTION_RUN_ID: ${{github.run_id}}
run: |
./tools/bin/gke-kube-acceptance-test/acceptance_test_kube_gke.sh
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,12 @@ jobs:
SENTRY_PROJECT: airbyte-connectors
- name: Publish ${{ github.event.inputs.connector }}
run: |
echo "$SPEC_CACHE_SERVICE_ACCOUNT_KEY" > spec_cache_key_file.json && docker login -u ${DOCKER_USERNAME} -p ${DOCKER_PASSWORD}
echo "$SPEC_CACHE_SERVICE_ACCOUNT_KEY" > spec_cache_key_file.json && docker login -u ${DOCKER_HUB_USERNAME} -p ${DOCKER_HUB_PASSWORD}
./tools/integrations/manage.sh publish airbyte-integrations/${{ github.event.inputs.connector }} ${{ github.event.inputs.run-tests }} --publish_spec_to_cache
id: publish
env:
DOCKER_USERNAME: airbytebot
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
# Oracle expects this variable to be set. Although usually present, this is not set by default on Github virtual runners.
TZ: UTC
- name: Finalize Sentry release
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/publish-external-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,12 +85,13 @@ jobs:
repository: ${{ gituhb.event.inputs.repo }}
ref: ${{ github.event.inputs.gitref }}
- run: |
echo "$SPEC_CACHE_SERVICE_ACCOUNT_KEY" > spec_cache_key_file.json && docker login -u airbytebot -p ${DOCKER_PASSWORD}
echo "$SPEC_CACHE_SERVICE_ACCOUNT_KEY" > spec_cache_key_file.json && docker login -u ${DOCKER_HUB_USERNAME} -p ${DOCKER_HUB_PASSWORD}
./tools/integrations/manage.sh publish_external ${{ github.event.inputs.connector }} ${{ github.event.inputs.version }}
name: publish ${{ github.event.inputs.connector }}
id: publish
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
# Oracle expects this variable to be set. Although usually present, this is not set by default on Github virtual runners.
TZ: UTC
- name: Add Success Comment
Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/release-airbyte-os.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ jobs:
- name: Release Airbyte
id: release_airbyte
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
PART_TO_BUMP: ${{ github.event.inputs.partToBump }}
CLOUDREPO_USER: ${{ secrets.CLOUDREPO_USER }}
CLOUDREPO_PASSWORD: ${{ secrets.CLOUDREPO_PASSWORD }}
Expand Down Expand Up @@ -100,7 +101,8 @@ jobs:
- name: Release Octavia
id: release_octavia
env:
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_PASSWORD: ${{ secrets.DOCKER_HUB_PASSWORD }}
PART_TO_BUMP: ${{ github.event.inputs.partToBump }}
run: ./tools/bin/release_version_octavia.sh

Expand Down
4 changes: 2 additions & 2 deletions tools/bin/gke-kube-acceptance-test/acceptance_test_kube_gke.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ echo "Namespace" $NAMESPACE
TAG=$(openssl rand -hex 12)
echo "Tag" $TAG

docker login -u airbytebot -p $DOCKER_PASSWORD
docker login -u "$DOCKER_HUB_USERNAME" -p "$DOCKER_HUB_PASSWORD"
VERSION=$TAG ./gradlew build
VERSION=$TAG docker-compose -f docker-compose.build.yaml push

Expand Down Expand Up @@ -44,7 +44,7 @@ function findAndDeleteTag () {
}

function cleanUpImages () {
TOKEN=$(curl --request POST 'https://hub.docker.com/v2/users/login/' --header 'Content-Type: application/json' --data-raw '{"username":"airbytebot","password":"'$DOCKER_PASSWORD'"}' | jq '.token')
TOKEN=$(curl --request POST 'https://hub.docker.com/v2/users/login/' --header 'Content-Type: application/json' --data-raw '{"username":"'$DOCKER_HUB_USERNAME'","password":"'$DOCKER_HUB_PASSWORD'"}' | jq '.token')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm - this one is also grabbing a token through curl, do we need to set +x set -x here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, because no where in the file do we turn that on (set +x)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... but maybe we should?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! I see, it was specifically enabled on manage.sh - should be ok then

TOKEN="${TOKEN%\"}"
TOKEN="${TOKEN#\"}"

Expand Down
11 changes: 8 additions & 3 deletions tools/bin/release_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,17 @@ if [[ -z "${CLOUDREPO_PASSWORD}" ]]; then
exit 1;
fi

if [[ -z "${DOCKER_PASSWORD}" ]]; then
echo 'DOCKER_PASSWORD for airbytebot not set.';
if [[ -z "${DOCKER_HUB_USERNAME}" ]]; then
echo 'DOCKER_HUB_USERNAME not set.';
exit 1;
fi

docker login -u airbytebot -p "${DOCKER_PASSWORD}"
if [[ -z "${DOCKER_HUB_PASSWORD}" ]]; then
echo 'DOCKER_HUB_PASSWORD for docker user not set.';
exit 1;
fi

docker login -u "${DOCKER_HUB_USERNAME}" -p "${DOCKER_HUB_PASSWORD}"

source ./tools/bin/bump_version.sh

Expand Down
11 changes: 8 additions & 3 deletions tools/bin/release_version_octavia.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,17 @@ set -e

. tools/lib/lib.sh

if test -z "${DOCKER_PASSWORD}"; then
echo 'DOCKER_PASSWORD for airbytebot not set.';
if test -z "${DOCKER_HUB_USERNAME}"; then
echo 'DOCKER_HUB_USERNNAME not set.';
exit 1;
fi

docker login --username airbytebot --password "${DOCKER_PASSWORD}"
if test -z "${DOCKER_HUB_PASSWORD}"; then
echo 'DOCKER_HUB_PASSWORD for docker user not set.';
exit 1;
fi

docker login --username "${DOCKER_HUB_USERNAME}" --password "${DOCKER_HUB_PASSWORD}"

source ./tools/bin/bump_version.sh

Expand Down
13 changes: 11 additions & 2 deletions tools/integrations/manage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,9 +213,18 @@ cmd_publish() {
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

# log into docker
DOCKER_USERNAME=${DOCKER_USERNAME:-airbytebot}
if test -z "${DOCKER_HUB_USERNAME}"; then
echo 'DOCKER_HUB_USERNAME not set.';
exit 1;
fi

if test -z "${DOCKER_HUB_PASSWORD}"; then
echo 'DOCKER_HUB_PASSWORD for docker user not set.';
exit 1;
fi

set +x
DOCKER_TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${DOCKER_USERNAME}'", "password": "'${DOCKER_PASSWORD}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)
DOCKER_TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${DOCKER_HUB_USERNAME}'", "password": "'${DOCKER_HUB_PASSWORD}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)
set -x

echo "image_name $image_name"
Expand Down