Skip to content

ci-operator/templates/openshift/installer/cluster-launch-installer-e2e: Gather console logs from Machine providerID too #6906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

wking
Copy link
Member

@wking wking commented Jan 29, 2020

To help debug cases like:

  spec:
    ...
    providerID: aws:///us-east-1b/i-078909486b3234e41
    ...
  status:
    ...
    phase: Provisioned
    ...

where the machine seems to have come up (per AWS), but not requested an Ignition config from the machine-config server, and therefore not joined the cluster as a node.

@openshift-ci-robot openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Jan 29, 2020
@wking wking force-pushed the console-logs-from-machine-instance-ids branch from 8b749c3 to a469f53 Compare January 29, 2020 20:39
@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 29, 2020
@hexfusion
Copy link
Contributor

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 29, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 9df63fc into openshift:master Jan 29, 2020
@openshift-ci-robot
Copy link
Contributor

@wking: Updated the following 3 configmaps:

  • prow-job-cluster-launch-installer-e2e configmap in namespace ci at cluster ci/api-build01-ci-devcluster-openshift-com:6443 using the following files:
    • key cluster-launch-installer-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-e2e.yaml
  • prow-job-cluster-launch-installer-e2e configmap in namespace ci at cluster default using the following files:
    • key cluster-launch-installer-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-e2e.yaml
  • prow-job-cluster-launch-installer-e2e configmap in namespace ci-stg at cluster default using the following files:
    • key cluster-launch-installer-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-e2e.yaml

In response to this:

To help debug cases like:

 spec:
   ...
   providerID: aws:///us-east-1b/i-078909486b3234e41
   ...
 status:
   ...
   phase: Provisioned
   ...

where the machine seems to have come up (per AWS), but not requested an Ignition config from the machine-config server, and therefore not joined the cluster as a node.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

@wking: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/rehearse/openshift/cloud-credential-operator/master/e2e-gcp a469f53 link /test pj-rehearse
ci/rehearse/openshift/cloud-credential-operator/master/e2e-azure a469f53 link /test pj-rehearse
ci/prow/pj-rehearse a469f53 link /test pj-rehearse

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@wking wking deleted the console-logs-from-machine-instance-ids branch February 16, 2020 16:53
wking added a commit to wking/openshift-release that referenced this pull request Jun 18, 2020
History of this logic:

* Initially landed for nodes in e102a16
  (ci-operator/templates/openshift/installer/cluster-launch-installer-e2e:
  Gather node console logs on AWS, 2019-12-02, openshift#6189).

* Grew --text in 6ec5bf3 (installer artifacts: keep text version of
  instance output, 2020-01-02, openshift#6536).

* Grew machine handling in a469f53
  (ci-operator/templates/openshift/installer/cluster-launch-installer-e2e:
  Gather console logs from Machine providerID too, 2020-01-29, openshift#6906).

* The node-provider-IDs creation was ported to steps in e2fd5c7
  (step-registry: update deprovision step, 2020-01-30, openshift#6708), but
  without any consumer for the collected file.

The aws-instance-ids.txt injection allows install-time steps to
register additional instances for later console collection (for a
proxy instance, bootstrap instance, etc.).

Approvers are:

* Myself and Vadim, who have touched this logic in the past.
* Alberto, representing the machine-API space that needs console logs
  to debug failed-to-boot issues.
* Colin, representing the RHCOS/machine-config space that needs
  console logs to debug RHCOS issues.

TMPDIR documentation is based on POSIX [1].

[1]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants