Update to Kind 1.27 and fix the integration tests

[juliusvonkohout]

Update to Kind 1.27 such that we have a supported upgrade path from 1.8.0 on Kubernetes 1.26 -> 1.8.1 on Kubernetes 1.26 -> 1.8.1 on Kubernetes 1.27 -> 1.9 on Kubernetes 1.27 -> Kubernetes 1.29

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[juliusvonkohout]

@andreyvelich in kubeflow/katib#2256 (comment) says that they already support 1.27. Maybe our tests are outdated

See also #2602

[juliusvonkohout]

manifests/.github/workflows/katib_kind_test.yaml

Lines 45 to 58 in 39bac59

	- name: Create katib experiment
	run: |
	kubectl create namespace kubeflow-user
	kubectl label namespace kubeflow-user katib.kubeflow.org/metrics-collector-injection=enabled
	kubectl apply -f tests/gh-actions/kf-objects/katib_test.yaml
	echo "Waiting for Experiment to become Running..."
	kubectl wait --for=condition=Running experiments.kubeflow.org -n kubeflow-user --all --timeout 300s
	echo "Waiting for all Trials to become Succeeded..."
	kubectl wait --for=condition=Succeeded trials.kubeflow.org -n kubeflow-user --all --timeout 300s
	echo "Waiting for the Experiment to become Succeeded..."
	kubectl wait --for=condition=Succeeded experiments.kubeflow.org -n kubeflow-user --all --timeout 300s

and or

https://github.com/kubeflow/manifests/blob/39bac597c9a0b3011db520e40b15f0974f765a0f/tests/gh-actions/kf-objects/katib_test.yaml

might have to be updated

[tenzen-y]

@andreyvelich in kubeflow/katib#2256 (comment) says that they already support 1.27. Maybe our tests are outdated

See also #2602

Yea, that's right. The katib already supports the K8s v1.27.

[juliusvonkohout]

@tenzen-y can you provide us a modern test file than that finishes in 2 minutes on Kubernetes 1.27?

[tenzen-y]

@tenzen-y can you provide us a modern test file than that finishes in 2 minutes on Kubernetes 1.27?

We're using the same test as #2637 (comment)

[tenzen-y]

@juliusvonkohout Oh, I found that this repository uses the docker.io/kubeflowkatib/mxnet-mnist:latest here:

manifests/tests/gh-actions/kf-objects/katib_test.yaml

Line 56 in 39bac59

image: docker.io/kubeflowkatib/mxnet-mnist:latest

But the image wouldn't work well since the MXNet project has already been closed. So, once kubeflow/katib#2267 is merged, I can share a new example with you.

[juliusvonkohout]

@tenzen-y i crafted something very fast from your latest PR, since the tests must run in 2 minutes. Seems to be good now.

[tenzen-y]

LGTM perspective of Katib.

[tenzen-y]

Suggested change

---

This is duplicated.