Skip to content

vsphere: make sure CSRs are approved, run imageregistry patch in a thread #3573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2019

Conversation

vrutkovs
Copy link
Member

This should stuck vsphere tests in installer/origin

@openshift-ci-robot openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 24, 2019
@droslean
Copy link
Member

/test pj-rehearse

@vrutkovs vrutkovs force-pushed the vsphere-csr-approval branch 3 times, most recently from b28d5a6 to cdfc36b Compare April 24, 2019 15:27
function approve_csrs() {
while true; do
sleep 5;
oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs --no-run-if-empty oc adm certificate approve && break
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we looping here because we have seen cases where the CSRs were not available yet? If the CSRs trickle in instead of all coming in at once, then this has the potential to approve some but not all of the CSRs. Should we keep approving CSRs until wait-for install-complete succeeds?

Copy link
Member Author

@vrutkovs vrutkovs Apr 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the idea is that by the time bootstrap is destroyed all nodes have created a CSR. Not sure if there is a simple way to run CSR approval loop until install-complete.

Previously xargs was accepting an empty list (probably api didn't return anything?) and proceeded. I haven't seen the case where just a part of nodes have submitted a CSR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we not create a file after wait-for install-complete and break out of the approve_csrs while loop when that file exists?

@vrutkovs vrutkovs force-pushed the vsphere-csr-approval branch from cdfc36b to 00d8d88 Compare April 24, 2019 16:28
@@ -427,17 +427,34 @@ objects:
terraform apply -auto-approve -var 'bootstrap_complete=true' -no-color &
wait "$!"

function approve_csrs() {
while true; do
if [[ ! -f /tmp/install-complete ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I reading this wrong or does this wait for the install-complete file to exist before approving the CSRs?

I was expecting something like the following.

function approve_csrs() {
  while [[ ! -f /tmp/install-complete ]]; do
    sleep 15;
    oc get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs --no-run-if-empty oc adm certificate approve
  done
}

Copy link
Member Author

@vrutkovs vrutkovs Apr 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, correct, this won't ever finish, as CSR should be approved earlier. Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we gaining by having an if statement that does a continue in one branch and a break in the other rather than moving the check up to the while statement?

@vrutkovs vrutkovs force-pushed the vsphere-csr-approval branch 2 times, most recently from f11b255 to 0f6142a Compare April 24, 2019 16:59
@vrutkovs vrutkovs force-pushed the vsphere-csr-approval branch from 0f6142a to 3dc122d Compare April 24, 2019 17:08
@vrutkovs
Copy link
Member Author

/test pj-rehearse

@openshift-ci-robot
Copy link
Contributor

@vrutkovs: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/rehearse/openshift/installer/master/e2e-vsphere 3dc122d link /test pj-rehearse
ci/prow/pj-rehearse 3dc122d link /test pj-rehearse

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link
Contributor

@staebler staebler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 24, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: staebler, vrutkovs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 24, 2019
@openshift-merge-robot openshift-merge-robot merged commit 1d0a3e2 into openshift:master Apr 24, 2019
@openshift-ci-robot
Copy link
Contributor

@vrutkovs: Updated the following 2 configmaps:

  • prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci using the following files:
    • key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml
  • prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci-stg using the following files:
    • key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml

In response to this:

This should stuck vsphere tests in installer/origin

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

abhinavdahiya added a commit to abhinavdahiya/release that referenced this pull request Apr 25, 2019
@vrutkovs vrutkovs deleted the vsphere-csr-approval branch January 27, 2020 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants