Skip to content

OPRUN-3962: Add structure to allow move the orgin tests using OTE #383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

camilamacedo86
Copy link
Contributor

@camilamacedo86 camilamacedo86 commented Jun 13, 2025

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=d02df2ca' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-07-01T16:42:33Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=dirty'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/operator-controller-tests-ext ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/olmv1-tests-ext info
{
    "apiVersion": "v1.1",
    "source": {
        "commit": "a43dca9a",
        "build_date": "2025-07-04T13:15:48Z",
        "git_tree_state": "clean"
    },
    "component": {
        "product": "openshift",
        "type": "payload",
        "name": "olmv1"
    },
    "suites": [
        {
            "name": "olmv1/parallel",
            "description": "",
            "parents": [
                "openshift/conformance/parallel"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:olmv1\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
            ]
        },
        {
            "name": "olmv1/serial",
            "description": "",
            "parents": [
                "openshift/conformance/serial"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:olmv1\") \u0026\u0026 (name.contains(\"[Serial]\"))"
            ]
        },
        {
            "name": "olmv1/slow",
            "description": "",
            "parents": [
                "openshift/optional/slow"
            ],
            "qualifiers": [
                "(source == \"openshift:payload:olmv1\") \u0026\u0026 (name.contains(\"[Slow]\"))"
            ]
        },
        {
            "name": "olmv1/all",
            "description": "",
            "parents": [
                "openshift/conformance/serial"
            ]
        }
    ],
    "images": null
}

  • Confirm list outputs tests properly filtered by environment
$ ./bin/operator-controller-tests-ext list
[
  {
    "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
    "labels": {},
    "resources": {
      "isolation": {}
    },
    "source": "olm:payload:olmv1-tests-extension",
    "codeLocations": [
      "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
      "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
    ],
    "lifecycle": "blocking",
    "environmentSelector": {}
  }
]

  • Confirm run-test produces valid JSONL test results
  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
 $ ./bin/olmv1-tests-ext run-suite olmv1/all
  Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
  ===============================================================================================================
  Random Seed: 1751635012 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [sig-olmv1] OLMv1 should pass a trivial sanity check
  /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-07-04 13:16:52.960749 UTC",
    "endTime": "2025-07-04 13:16:52.961643 UTC",
    "result": "passed",
    "output": ""
  }
]

./my-component-tests run-test -n

$ ./bin/olmv1-tests-ext run-test -n "[sig-olmv1] OLMv1 should pass a trivial sanity check"
  Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
  ===============================================================================================================
  Random Seed: 1751635078 - will randomize all specs

  Will run 1 of 1 specs
  ------------------------------
  [sig-olmv1] OLMv1 should pass a trivial sanity check
  /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
  • [0.000 seconds]
  ------------------------------

  Ran 1 of 1 Specs in 0.000 seconds
  SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
  {
    "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check",
    "lifecycle": "blocking",
    "duration": 0,
    "startTime": "2025-07-04 13:17:58.677690 UTC",
    "endTime": "2025-07-04 13:17:58.678274 UTC",
    "result": "passed",
    "output": ""
  }
]

Blocks

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 13, 2025
@camilamacedo86 camilamacedo86 changed the title WIP Add OTE for origin tests WIP OPRUN-3962: Add OTE for origin tests Jun 13, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 13, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 13, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86 camilamacedo86 force-pushed the extended-tests-bin branch 2 times, most recently from 5893e18 to 039e348 Compare June 30, 2025 19:09
@camilamacedo86 camilamacedo86 changed the title WIP OPRUN-3962: Add OTE for origin tests OPRUN-3962: Add OTE for origin tests Jun 30, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 30, 2025
@camilamacedo86 camilamacedo86 changed the title OPRUN-3962: Add OTE for origin tests OPRUN-3962: Add structure to allow move the orgin tests using OTE Jun 30, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 30, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata

  • Confirm list outputs tests properly filtered by environment

  • Confirm run-test produces valid JSONL test results

  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
    ./my-component-tests run-test -n

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 30, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata

  • Confirm list outputs tests properly filtered by environment

  • Confirm run-test produces valid JSONL test results

  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
    ./my-component-tests run-test -n

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 30, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=039e348a' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-06-30T19:15:25Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=clean'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/tests-extension ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/tests-extension  info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "",
       "build_date": "",
       "git_tree_state": ""
   },
   "component": {
       "product": "operator-framework",
       "type": "payload",
       "name": "olmv1-tests-extension"
   },
   "suites": [
       {
           "name": "olmv1/tests",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ]
       }
   ],
   "images": null
}
  • Confirm list outputs tests properly filtered by environment
$ ./bin/tests-extension list
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "operator-framework:payload:olmv1-tests-extension",
   "codeLocations": [
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]
  • Confirm run-test produces valid JSONL test results

  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite

$ ./bin/tests-extension  run-suite olmv1/tests
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751311042 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.001 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 1,
   "startTime": "2025-06-30 19:17:22.643924 UTC",
   "endTime": "2025-06-30 19:17:22.644987 UTC",
   "result": "passed",
   "output": ""
 }
]

./my-component-tests run-test -n

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 30, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=039e348a' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-06-30T19:15:25Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=clean'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/tests-extension ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/tests-extension  info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "",
       "build_date": "",
       "git_tree_state": ""
   },
   "component": {
       "product": "operator-framework",
       "type": "payload",
       "name": "olmv1-tests-extension"
   },
   "suites": [
       {
           "name": "olmv1/tests",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ]
       }
   ],
   "images": null
}
  • Confirm list outputs tests properly filtered by environment
$ ./bin/tests-extension list
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "operator-framework:payload:olmv1-tests-extension",
   "codeLocations": [
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]
  • Confirm run-test produces valid JSONL test results
  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
$ ./bin/tests-extension  run-suite olmv1/tests
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751311042 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.001 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 1,
   "startTime": "2025-06-30 19:17:22.643924 UTC",
   "endTime": "2025-06-30 19:17:22.644987 UTC",
   "result": "passed",
   "output": ""
 }
]

./my-component-tests run-test -n

$ ./bin/tests-extension run-test -n "[sig-operator] OLMv1 should pass a trivial sanity check"
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751311251 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-06-30 19:20:51.745159 UTC",
   "endTime": "2025-06-30 19:20:51.745651 UTC",
   "result": "passed",
   "output": ""
 }
]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86 camilamacedo86 force-pushed the extended-tests-bin branch 2 times, most recently from 8bf2366 to c7c197d Compare June 30, 2025 19:35
@kuiwang02
Copy link

@camilamacedo86 thanks. I will review it.

@kuiwang02
Copy link

I will finish review tomorrow because I need to take more time review it.
please keep the PR unchanged, thanks

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 1, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=d02df2ca' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-07-01T16:42:33Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=dirty'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/operator-controller-tests-ext ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/operator-controller-tests-ext info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "",
       "build_date": "",
       "git_tree_state": ""
   },
   "component": {
       "product": "olm",
       "type": "payload",
       "name": "olmv1-tests-extension"
   },
   "suites": [
       {
           "name": "olmv1/tests",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ]
       }
   ],
   "images": null
}

  • Confirm list outputs tests properly filtered by environment
$ ./bin/operator-controller-tests-ext list
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "olm:payload:olmv1-tests-extension",
   "codeLocations": [
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]

  • Confirm run-test produces valid JSONL test results
  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
./bin/operator-controller-tests-ext run-suite olmv1/tests
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751388026 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-01 16:40:26.670869 UTC",
   "endTime": "2025-07-01 16:40:26.671756 UTC",
   "result": "passed",
   "output": ""
 }
]

./my-component-tests run-test -n

$ ./bin/operator-controller-tests-ext run-test -n "[sig-operator] OLMv1 should pass a trivial sanity check" 
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751388091 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-01 16:41:31.679220 UTC",
   "endTime": "2025-07-01 16:41:31.679747 UTC",
   "result": "passed",
   "output": ""
 }
]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86 camilamacedo86 force-pushed the extended-tests-bin branch 2 times, most recently from f8cf6f6 to bd7b074 Compare July 1, 2025 16:51
@camilamacedo86
Copy link
Contributor Author

Hi @kuiwang02

@camilamacedo86 thanks. I will review it.
I will finish review tomorrow because I need to take more time review it.
please keep the PR unchanged, thanks

Thank you for checking the PR! 😊

I needed to change it because now we can use Go 1.24. So, I rebased it and also renamed the bin to match the others — like in this example: https://github.com/openshift/origin/pull/29949/files.

Please note: this PR only adds the bin. There are no tests.
To try it, just pull the PR and run make build — that will give you the bin locally.
(You can also check the PR description.)

I didn’t understand the concern, since we are not using OCP, mocks, or special test cases here.
I just wanted to let you know, so you can fetch it again if needed.

Thanks a lot for your help and understanding! 🙏

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 1, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=d02df2ca' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-07-01T16:42:33Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=dirty'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/operator-controller-tests-ext ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/operator-controller-tests-ext info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "",
       "build_date": "",
       "git_tree_state": ""
   },
   "component": {
       "product": "olm",
       "type": "payload",
       "name": "olmv1-tests-extension"
   },
   "suites": [
       {
           "name": "olmv1/tests",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ]
       }
   ],
   "images": null
}

  • Confirm list outputs tests properly filtered by environment
$ ./bin/operator-controller-tests-ext list
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "olm:payload:olmv1-tests-extension",
   "codeLocations": [
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]

  • Confirm run-test produces valid JSONL test results
  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
./bin/operator-controller-tests-ext run-suite olmv1/tests
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751388026 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-01 16:40:26.670869 UTC",
   "endTime": "2025-07-01 16:40:26.671756 UTC",
   "result": "passed",
   "output": ""
 }
]

./my-component-tests run-test -n

$ ./bin/operator-controller-tests-ext run-test -n "[sig-operator] OLMv1 should pass a trivial sanity check" 
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751388091 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-operator] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-01 16:41:31.679220 UTC",
   "endTime": "2025-07-01 16:41:31.679747 UTC",
   "result": "passed",
   "output": ""
 }
]

Blocks

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

@anik120 anik120 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check in the tests/extension/vendor folder? I understand that we are checking in the top level vendor folder for this repo, but that's probably because of some downstream-ing process quirks.

It'll be cleaner to have the vendor folder git ignored, and then the Make recipes to include go mod vendor before building the binary, if possible

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 4, 2025

@camilamacedo86: This pull request references OPRUN-3962 which is a valid jira issue.

In response to this:

Description

This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.

Checklist from https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0

  • Setup project with Dockerfile for CI
  • Initialize the extension
  • Build the binary (make build)
$ make build
# GO_COMPLIANCE_POLICY="exempt_all" must only be used for test related binaries.
# It prevents various FIPS compliance policies from being applied to this compilation.
# Do not set globally.
GO_COMPLIANCE_POLICY="exempt_all" go build -ldflags "-X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.CommitFromGit=d02df2ca' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.BuildDate=2025-07-01T16:42:33Z' -X 'github/operator-framework-operator-controller/openshift/extended-tests/pkg/version.GitTreeState=dirty'" -o /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/bin/operator-controller-tests-ext ./cmd/...
  • Test locally (following bellow)

Local Tests

  • Confirm info outputs correct API version, metadata
$ ./bin/olmv1-tests-ext info
{
   "apiVersion": "v1.1",
   "source": {
       "commit": "a43dca9a",
       "build_date": "2025-07-04T13:15:48Z",
       "git_tree_state": "clean"
   },
   "component": {
       "product": "openshift",
       "type": "payload",
       "name": "olmv1"
   },
   "suites": [
       {
           "name": "olmv1/parallel",
           "description": "",
           "parents": [
               "openshift/conformance/parallel"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:olmv1\") \u0026\u0026 (!(name.contains(\"[Serial]\") || name.contains(\"[Slow]\")))"
           ]
       },
       {
           "name": "olmv1/serial",
           "description": "",
           "parents": [
               "openshift/conformance/serial"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:olmv1\") \u0026\u0026 (name.contains(\"[Serial]\"))"
           ]
       },
       {
           "name": "olmv1/slow",
           "description": "",
           "parents": [
               "openshift/optional/slow"
           ],
           "qualifiers": [
               "(source == \"openshift:payload:olmv1\") \u0026\u0026 (name.contains(\"[Slow]\"))"
           ]
       },
       {
           "name": "olmv1/all",
           "description": "",
           "parents": [
               "openshift/conformance/serial"
           ]
       }
   ],
   "images": null
}

  • Confirm list outputs tests properly filtered by environment
$ ./bin/operator-controller-tests-ext list
[
 {
   "name": "[sig-operator] OLMv1 should pass a trivial sanity check",
   "labels": {},
   "resources": {
     "isolation": {}
   },
   "source": "olm:payload:olmv1-tests-extension",
   "codeLocations": [
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
     "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
   ],
   "lifecycle": "blocking",
   "environmentSelector": {}
 }
]

  • Confirm run-test produces valid JSONL test results
  • Confirm you can run suites and tests locally:
    ./my-component-tests run-suite
$ ./bin/olmv1-tests-ext run-suite olmv1/all
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751635012 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-olmv1] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-04 13:16:52.960749 UTC",
   "endTime": "2025-07-04 13:16:52.961643 UTC",
   "result": "passed",
   "output": ""
 }
]

./my-component-tests run-test -n

$ ./bin/olmv1-tests-ext run-test -n "[sig-olmv1] OLMv1 should pass a trivial sanity check"
 Running Suite:  - /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension
 ===============================================================================================================
 Random Seed: 1751635078 - will randomize all specs

 Will run 1 of 1 specs
 ------------------------------
 [sig-olmv1] OLMv1 should pass a trivial sanity check
 /Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13
 • [0.000 seconds]
 ------------------------------

 Ran 1 of 1 Specs in 0.000 seconds
 SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
[
 {
   "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check",
   "lifecycle": "blocking",
   "duration": 0,
   "startTime": "2025-07-04 13:17:58.677690 UTC",
   "endTime": "2025-07-04 13:17:58.678274 UTC",
   "result": "passed",
   "output": ""
 }
]

Blocks

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86
Copy link
Contributor Author

@anik120 Can you please unblock this PR?
See #383 (comment)
And #383 (comment)

We must have the vendor so far.

Copy link
Contributor

@anik120 anik120 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86 makes sense, thanks for checking that and confirming.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2025
@kuiwang02
Copy link

@camilamacedo86 makes sense, thanks for checking that and confirming.

/lgtm

reminder:
before merging PR, need to resolve #383 (comment).

@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 8, 2025
This commit introduces a binary and supporting structure to enable the
execution of OpenShift origin (olmv1) tests using the Open Test Environment (OTE).

It lays the groundwork for moving origin test in openshift/origin to
be executed from this repository using OTE.
@camilamacedo86
Copy link
Contributor Author

/test okd-scos-e2e-aws-ovn

Copy link
Contributor

openshift-ci bot commented Jul 8, 2025

@camilamacedo86: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@perdasilva
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 9, 2025
Copy link
Contributor

openshift-ci bot commented Jul 9, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: anik120, camilamacedo86, perdasilva

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 9, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit ad904c5 into openshift:main Jul 9, 2025
10 checks passed
@camilamacedo86 camilamacedo86 deleted the extended-tests-bin branch July 9, 2025 11:44
@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-catalogd
This PR has been included in build ose-olm-catalogd-container-v4.20.0-202507091614.p0.gad904c5.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-operator-controller
This PR has been included in build ose-olm-operator-controller-container-v4.20.0-202507091614.p0.gad904c5.assembly.stream.el9.
All builds following this will include this PR.

camilamacedo86 added a commit to camilamacedo86/operator-framework-operator-controller that referenced this pull request Jul 10, 2025
openshift-merge-bot bot pushed a commit that referenced this pull request Jul 11, 2025
openshift-bot pushed a commit to openshift-bot/operator-framework-operator-controller that referenced this pull request Jul 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants