config: add periodic job that runs upgrades using multistage tests #9489

AlexNPavel · 2020-06-04T18:47:41Z

Add new periodic that runs upgrade test from 4.4 stable to 4.5 nightly using multistage tests.

/cc @stevekuznetsov @bbguimaraes

openshift-ci-robot · 2020-06-04T18:47:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AlexNPavel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~ci-operator/config/openshift/release/OWNERS~~ [AlexNPavel]
~~ci-operator/jobs/openshift/release/OWNERS~~ [AlexNPavel]
~~ci-operator/step-registry/ipi/conf/OWNERS~~ [AlexNPavel]
~~ci-operator/step-registry/ipi/install/install/OWNERS~~ [AlexNPavel]
~~ci-operator/step-registry/openshift/e2e/test/OWNERS~~ [AlexNPavel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

AlexNPavel · 2020-06-04T21:29:16Z

/retest

AlexNPavel · 2020-06-04T23:37:20Z

/test all

AlexNPavel · 2020-06-05T17:56:29Z

/hold
/test all

stevekuznetsov · 2020-06-05T18:06:05Z

ci-operator/config/openshift/release/openshift-release-master__ocp-4.5-upgrades.yaml

@@ -0,0 +1,30 @@
+releases:


Why did you need this upgrade variant config?

Mainly to make it as close to identical as I can to the existing upgrade tests for 4.5. Those all go 4.4-stable to 4.5.0-0.ci: https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/release/openshift-release-release-4.5-periodics.yaml#L5556

Not sure if the difference between nightly and ci really matters much for the periodic upgrade tests though.

We do want them to be identical

I guess we will want a set of ocp-4.5-ci and ocp-4.5-nightly and ocp-4.5-prerelease variants (defined by what it means to be latest in each config)

Renamed the new variant to ocp-4.5-ci

stevekuznetsov · 2020-06-05T21:30:12Z

ci-operator/config/openshift/release/openshift-release-master__ocp-4.5-upgrades.yaml

+      cpu: 100m
+      memory: 200Mi
+tests:
+- as: e2e-upgrade-informer


Why isn't there an OPENSHIFT_UPGRADE=true env here?

Now it's set.

stevekuznetsov · 2020-06-05T21:30:59Z

ci-operator/step-registry/ipi/conf/ipi-conf-commands.sh

@@ -13,7 +13,11 @@ if [[ -z "$RELEASE_IMAGE_LATEST" ]]; then
  exit 1
 fi

-echo "Installing from release ${RELEASE_IMAGE_LATEST}"
+if [[ "${OPENSHIFT_UPGRADE:-false}" == "false" ]]; then


Why do we need this here? I'd rather just remove it and have us print this during the install step, no need to have conf care if it does nothing with this value

Moved the echo to the install step.

AlexNPavel · 2020-06-05T22:58:30Z

/test pj-rehearse

AlexNPavel · 2020-06-08T17:38:29Z

/test core-valid

AlexNPavel · 2020-06-08T18:37:12Z

/test pj-rehearse

AlexNPavel · 2020-06-08T22:39:37Z

/test pj-rehearse

wking · 2020-06-15T23:34:55Z

ci-operator/step-registry/ipi/conf/ipi-conf-commands.sh

@@ -13,8 +13,6 @@ if [[ -z "$RELEASE_IMAGE_LATEST" ]]; then
  exit 1
 fi

-echo "Installing from release ${RELEASE_IMAGE_LATEST}"


This seems orthogonal to the thrust of this PR and the echo dates back to ipi-conf-commands.sh from #6708. Can we spin this hunk out into a separate PR and get it landed while this one gets iterated on?

Ah, reading down I see you adding a similar echo in ipi-install-install-commands.sh, so I'm meh on whether we pull this out into a separate PR or not. And while I'm talking about ipi-install-install-commands.sh, seems like a chance to remind folks that the stutter-removing PR is alive again: #7931.

wking · 2020-06-15T23:36:13Z

ci-operator/step-registry/ipi/conf/ipi-conf-ref.yaml

+  from_image:
+    namespace: origin
+    name: centos
+    tag: '8'


Why this change? base seems less opinionated for steps that just need generic POSIX stuff, and it's one less thing to forget to bump.

wking · 2020-06-15T23:42:13Z

ci-operator/step-registry/ipi/install/install/ipi-install-install-ref.yaml

@@ -6,5 +6,8 @@ ref:
    requests:
      cpu: 1000m
      memory: 2Gi
+  env:
+  - name: OPENSHIFT_UPGRADE
+    default: "false"


Please add documentation :). Something like:

Select between ${RELEASE_IMAGE_INITIAL} (true) or ${RELEASE_IMAGE_LATEST} (false) for the installed version.

Not clear to me if documentation supports Markdown. If so, backtick the environment variables?

wking · 2020-06-16T03:21:19Z

ci-operator/step-registry/openshift/e2e/test/openshift-e2e-test-commands.sh

-
-openshift-tests run "${test_suite}" \
-    --provider "${TEST_PROVIDER}" \
+if [[ "${OPENSHIFT_UPGRADE:-false}" == "false" ]]; then


Hrm. Not clear to me how much we want to pack into OPENSHIFT_UPGRADE here with an opinionated conditional inside the step. How about #9676 and then setting:

OPENSHIFT_UPGRADE=true # for the install switch. TEST_COMMAND=run-upgrade TEST_SUITE=all

Or maybe even RELEASE_IMAGE=RELEASE_IMAGE_INITIAL and then use indirect expansion:

$ RELEASE_IMAGE_INITIAL=A $ RELEASE_IMAGE_LATEST=B $ RELEASE_IMAGE=RELEASE_IMAGE_INITIAL $ echo "${!RELEASE_IMAGE}" A

And then setting those variables appropriately when we define the job?

ingvagabund · 2020-06-24T12:13:47Z

@AlexNPavel is this PR still active?

hexfusion · 2020-06-24T12:15:17Z

etcd-team is very interested in this for resolving blocker bugs

openshift-ci-robot · 2020-07-30T15:01:50Z

@AlexNPavel: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/rehearse/periodic-ci-openshift-release-master-ocp-4.5-ci-e2e-upgrade-informer	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/installer/master/e2e-gcp-upi	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/opendatahub-io/odh-manifests/master/odh-manifests-e2e	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-network-operator/master/e2e-windows-hybrid-network	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/open-cluster-management/registration-operator/master/e2e	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-network-operator/master/e2e-ovn-hybrid-step-registry	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/cri-o/cri-o/release-1.13/e2e-aws	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-network-operator/master/e2e-aws-sdn-single	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/installer/master/e2e-azure	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-network-operator/master/e2e-vsphere	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/installer/master/e2e-vsphere	`6b00ac3`	link	`/test pj-rehearse`
ci/rehearse/openshift/cluster-network-operator/master/e2e-ovn-step-registry	`6b00ac3`	link	`/test pj-rehearse`
ci/prow/pj-rehearse	`6b00ac3`	link	`/test pj-rehearse`
ci/prow/release-controller-config	`6b00ac3`	link	`/test release-controller-config`
ci/prow/step-registry-metadata	`6b00ac3`	link	`/test step-registry-metadata`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

stevekuznetsov · 2020-08-26T04:57:22Z

superseded by #11247

/close

openshift-ci-robot · 2020-08-26T04:57:37Z

@stevekuznetsov: Closed this PR.

In response to this:

superseded by #11247

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot requested review from bbguimaraes and stevekuznetsov June 4, 2020 18:47

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2020

AlexNPavel force-pushed the release-informing-step-registry branch 2 times, most recently from f0410cd to 78e6bb3 Compare June 4, 2020 18:55

AlexNPavel force-pushed the release-informing-step-registry branch 3 times, most recently from 48cb21e to e679654 Compare June 4, 2020 23:26

AlexNPavel marked this pull request as draft June 4, 2020 23:26

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 4, 2020

AlexNPavel force-pushed the release-informing-step-registry branch from e679654 to 803826a Compare June 4, 2020 23:29

AlexNPavel force-pushed the release-informing-step-registry branch from 803826a to 447deef Compare June 5, 2020 17:31

AlexNPavel marked this pull request as ready for review June 5, 2020 17:56

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 5, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 5, 2020

AlexNPavel force-pushed the release-informing-step-registry branch from 447deef to 6a8a6b4 Compare June 5, 2020 17:59

stevekuznetsov reviewed Jun 5, 2020

View reviewed changes

AlexNPavel force-pushed the release-informing-step-registry branch 8 times, most recently from eefd663 to 57192ce Compare June 5, 2020 21:00

stevekuznetsov reviewed Jun 5, 2020

View reviewed changes

AlexNPavel force-pushed the release-informing-step-registry branch from 57192ce to e2eec0e Compare June 5, 2020 22:03

AlexNPavel force-pushed the release-informing-step-registry branch from e2eec0e to 594d0fb Compare June 8, 2020 17:37

AlexNPavel force-pushed the release-informing-step-registry branch 2 times, most recently from 265935d to 4630fa6 Compare June 8, 2020 23:14

config: add periodic job that runs upgrades using multistage tests

6b00ac3

AlexNPavel force-pushed the release-informing-step-registry branch from 4630fa6 to 6b00ac3 Compare June 8, 2020 23:30

wking reviewed Jun 15, 2020

View reviewed changes

wking reviewed Jun 16, 2020

View reviewed changes

ingvagabund mentioned this pull request Jul 27, 2020

Deploy loki in a new template #10397

Closed

openshift-ci-robot closed this Aug 26, 2020

petr-muller mentioned this pull request Aug 26, 2020

step-registry: add upgrade workflows #11247

Merged

AlexNPavel deleted the release-informing-step-registry branch February 3, 2023 21:57

config: add periodic job that runs upgrades using multistage tests #9489

config: add periodic job that runs upgrades using multistage tests #9489

Uh oh!

Conversation

AlexNPavel commented Jun 4, 2020

Uh oh!

openshift-ci-robot commented Jun 4, 2020

Uh oh!

AlexNPavel commented Jun 4, 2020

Uh oh!

AlexNPavel commented Jun 4, 2020

Uh oh!

AlexNPavel commented Jun 5, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexNPavel commented Jun 5, 2020

Uh oh!

AlexNPavel commented Jun 8, 2020

Uh oh!

AlexNPavel commented Jun 8, 2020

Uh oh!

AlexNPavel commented Jun 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wking Jun 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wking Jun 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ingvagabund commented Jun 24, 2020

Uh oh!

hexfusion commented Jun 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Jul 30, 2020

Uh oh!

stevekuznetsov commented Aug 26, 2020

Uh oh!

openshift-ci-robot commented Aug 26, 2020

Uh oh!

Uh oh!

wking Jun 15, 2020 •

edited

Loading

wking Jun 16, 2020 •

edited

Loading

hexfusion commented Jun 24, 2020 •

edited

Loading