Skip to content

config: add periodic job that runs upgrades using multistage tests #9489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

config: add periodic job that runs upgrades using multistage tests #9489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions ci-operator/config/openshift/release/openshift-release-master__ocp-4.5-ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
releases:
initial:
release:
channel: stable
version: "4.4"
latest:
candidate:
product: ocp
stream: ci
version: "4.5"
resources:
'*':
requests:
cpu: 100m
memory: 200Mi
tests:
- as: e2e-upgrade-informer
cron: 0 */6 * * *
steps:
cluster_profile: aws
env:
OPENSHIFT_UPGRADE: "true"
workflow: openshift-e2e-aws
zz_generated_metadata:
branch: master
org: openshift
repo: release
variant: ocp-4.5-ci
81 changes: 81 additions & 0 deletions ci-operator/jobs/openshift/release/openshift-release-master-periodics.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,87 @@ periodics:
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: api.ci
cron: 0 */6 * * *
decorate: true
decoration_config:
skip_cloning: true
extra_refs:
- base_ref: master
org: openshift
repo: release
labels:
ci-operator.openshift.io/prowgen-controlled: "true"
ci-operator.openshift.io/variant: ocp-4.5-ci
pj-rehearse.openshift.io/can-be-rehearsed: "true"
name: periodic-ci-openshift-release-master-ocp-4.5-ci-e2e-upgrade-informer
spec:
containers:
- args:
- --image-import-pull-secret=/etc/pull-secret/.dockerconfigjson
- --kubeconfig=/etc/apici/kubeconfig
- --lease-server-password-file=/etc/boskos/password
- --report-password-file=/etc/report/password.txt
- --report-username=ci
- --secret-dir=/secrets/ci-pull-credentials
- --secret-dir=/usr/local/e2e-upgrade-informer-cluster-profile
- --target=e2e-upgrade-informer
- --variant=ocp-4.5-ci
command:
- ci-operator
image: ci-operator:latest
imagePullPolicy: Always
name: ""
resources:
requests:
cpu: 10m
volumeMounts:
- mountPath: /etc/apici
name: apici-ci-operator-credentials
readOnly: true
- mountPath: /etc/boskos
name: boskos
readOnly: true
- mountPath: /secrets/ci-pull-credentials
name: ci-pull-credentials
readOnly: true
- mountPath: /usr/local/e2e-upgrade-informer-cluster-profile
name: cluster-profile
- mountPath: /etc/pull-secret
name: pull-secret
readOnly: true
- mountPath: /etc/report
name: result-aggregator
readOnly: true
serviceAccountName: ci-operator
volumes:
- name: apici-ci-operator-credentials
secret:
items:
- key: sa.ci-operator.apici.config
path: kubeconfig
secretName: apici-ci-operator-credentials
- name: boskos
secret:
items:
- key: password
path: password
secretName: boskos-credentials
- name: ci-pull-credentials
secret:
secretName: ci-pull-credentials
- name: cluster-profile
projected:
sources:
- secret:
name: cluster-secrets-aws
- name: pull-secret
secret:
secretName: regcred
- name: result-aggregator
secret:
secretName: result-aggregator
- agent: kubernetes
cluster: api.ci
cron: 0 0 * * 1
Expand Down
5 changes: 4 additions & 1 deletion ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
ref:
as: ipi-conf-aws
from: base
from_image:
namespace: origin
name: centos
tag: '8'
commands: ipi-conf-aws-commands.sh
resources:
requests:
Expand Down
2 changes: 0 additions & 2 deletions ci-operator/step-registry/ipi/conf/ipi-conf-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ if [[ -z "$RELEASE_IMAGE_LATEST" ]]; then
exit 1
fi

echo "Installing from release ${RELEASE_IMAGE_LATEST}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems orthogonal to the thrust of this PR and the echo dates back to ipi-conf-commands.sh from #6708. Can we spin this hunk out into a separate PR and get it landed while this one gets iterated on?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, reading down I see you adding a similar echo in ipi-install-install-commands.sh, so I'm meh on whether we pull this out into a separate PR or not. And while I'm talking about ipi-install-install-commands.sh, seems like a chance to remind folks that the stutter-removing PR is alive again: #7931.


ssh_pub_key=$(<"${CLUSTER_PROFILE_DIR}/ssh-publickey")
pull_secret=$(<"${CLUSTER_PROFILE_DIR}/pull-secret")

Expand Down
5 changes: 4 additions & 1 deletion ci-operator/step-registry/ipi/conf/ipi-conf-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
ref:
as: ipi-conf
from: base
from_image:
namespace: origin
name: centos
tag: '8'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change? base seems less opinionated for steps that just need generic POSIX stuff, and it's one less thing to forget to bump.

commands: ipi-conf-commands.sh
resources:
requests:
Expand Down
7 changes: 6 additions & 1 deletion ci-operator/step-registry/ipi/install/install/ipi-install-install-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,12 @@ fi

export SSH_PRIV_KEY_PATH=${CLUSTER_PROFILE_DIR}/ssh-privatekey
export PULL_SECRET_PATH=${CLUSTER_PROFILE_DIR}/pull-secret
export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=${RELEASE_IMAGE_LATEST}
if [[ "${OPENSHIFT_UPGRADE:-false}" == "false" ]]; then
export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=${RELEASE_IMAGE_LATEST}
else
export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=${RELEASE_IMAGE_INITIAL}
fi
echo "Installing from release ${OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE}"
export OPENSHIFT_INSTALL_INVOKER=openshift-internal-ci/${JOB_NAME}/${BUILD_ID}
export HOME=/tmp

Expand Down
3 changes: 3 additions & 0 deletions ci-operator/step-registry/ipi/install/install/ipi-install-install-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,8 @@ ref:
requests:
cpu: 1000m
memory: 2Gi
env:
- name: OPENSHIFT_UPGRADE
default: "false"
Copy link
Member

@wking wking Jun 15, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add documentation :). Something like:

Select between ${RELEASE_IMAGE_INITIAL} (true) or ${RELEASE_IMAGE_LATEST} (false) for the installed version.

Not clear to me if documentation supports Markdown. If so, backtick the environment variables?

documentation: |-
The IPI install step runs the OpenShift Installer in order to bring up an OpenShift cluster, using the provided cluster profile to choose a target IaaS platform. Anything that needs to be configured using manifests should generate them before this step and put them in the SHARED_DIR with the filename manifest_(manifest_name).yml so that this step can pull in the manifest in.
25 changes: 18 additions & 7 deletions ci-operator/step-registry/openshift/e2e/test/openshift-e2e-test-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,23 @@ if [[ "${CLUSTER_TYPE}" == gcp ]]; then
popd
fi

test_suite=openshift/conformance/parallel
if [[ -e "${SHARED_DIR}/test-suite.txt" ]]; then
test_suite=$(<"${SHARED_DIR}/test-suite.txt")
fi

openshift-tests run "${test_suite}" \
--provider "${TEST_PROVIDER}" \
if [[ "${OPENSHIFT_UPGRADE:-false}" == "false" ]]; then
Copy link
Member

@wking wking Jun 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm. Not clear to me how much we want to pack into OPENSHIFT_UPGRADE here with an opinionated conditional inside the step. How about #9676 and then setting:

OPENSHIFT_UPGRADE=true  # for the install switch.
TEST_COMMAND=run-upgrade
TEST_SUITE=all

Or maybe even RELEASE_IMAGE=RELEASE_IMAGE_INITIAL and then use indirect expansion:

$ RELEASE_IMAGE_INITIAL=A
$ RELEASE_IMAGE_LATEST=B
$ RELEASE_IMAGE=RELEASE_IMAGE_INITIAL
$ echo "${!RELEASE_IMAGE}"
A

And then setting those variables appropriately when we define the job?

test_suite=openshift/conformance/parallel
if [[ -e "${SHARED_DIR}/test-suite.txt" ]]; then
test_suite=$(<"${SHARED_DIR}/test-suite.txt")
fi
openshift-tests run "${test_suite}" \
--provider "${TEST_PROVIDER}" \
-o /tmp/artifacts/e2e.log \
--junit-dir /tmp/artifacts/junit
else
test_suite=all
if [[ -e "${SHARED_DIR}/test-suite.txt" ]]; then
test_suite=$(<"${SHARED_DIR}/test-suite.txt")
fi
openshift-tests run-upgrade "${test_suite}" \
--to-image "${IMAGE:-${RELEASE_IMAGE_LATEST}}" \
--provider "${TEST_PROVIDER:-}" \
-o /tmp/artifacts/e2e.log \
--junit-dir /tmp/artifacts/junit
fi
3 changes: 3 additions & 0 deletions ci-operator/step-registry/openshift/e2e/test/openshift-e2e-test-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,8 @@ ref:
memory: 600Mi
limits:
memory: 4Gi
env:
- name: OPENSHIFT_UPGRADE
default: "false"
documentation: |-
The Openshift E2E step executes the common end-to-end test suite.