[Roadmap] KubeRay (or anything for Ray on K8s) v1.5.0 Wishlist

[kevin85421]

What feature do you want to have in KubeRay or Ray? Please add an emoji to the following comments if you find them useful. Please briefly explain the feature you want in a single comment. This issue is not for discussion, only for voting and proposing. For discussion, send a message to #kuberay-discuss.

[kevin85421]

Make Autoscaler V2 to be the default autoscaler option:

• [Umbrella] Autoscaler improvements #2600
• It should have better stability and observability compared to v1 after fixing issues.
• Run Autoscaler V2 in a separate Pod instead of a container in the head Pod.

[kevin85421]

RayService incremental upgrade:

• REP: KubeRay RayService Incremental Rollout enhancements#58
• This avoids 2X computing resources during the zero downtime upgrade process.

[kevin85421]

[Done: v1.4.0] Standardize KubeRay API server

• We found that some users built their own KubeRay API server. Standardizing the KubeRay API server speeds up future user adoption because they won't need to build their own KubeRay API server again.
• Make KubeRay API server flexible: Currently, the KubeRay API server interface is not flexible. Users need to open a PR if a field is not exposed.

[kevin85421]

Idle cluster termination: #2998

• Terminate a RayCluster if there is no running Ray job to save $$$$.

[kevin85421]

Documentation and Terraform for the reference architecture

• For example, GPU scheduling, reduce image pulling overhead, logging, notifications, ... etc

[kevin85421]

Light-weight job submitter:

• [Feature] Light-weight job submitter #2537
• This allows the K8s job submitter to avoid pulling the Ray image, which is typically over 1 GB even in its thinnest version without ML libraries. The light-weight job submitter I expected to be less than 20 MB. This will enhance the startup time of RayJob.

[kevin85421]

Integrate Volcano with RayJob: currently, Volcano only integrates with RayCluster.

[kevin85421]

Integrate YuniKorn with RayJob: currently, YuniKorn only integrates with RayCluster.

[kevin85421]

Support cron scheduling in RayJob

• [Feature] Support cron scheduling for RayJob #2426

[kevin85421]

[Done: v1.4.0] KubeRay dashboard:

• A frontend to visualize and manage (e.g. create / delete) KubeRay custom resources.
• Something similar to the frontend from Roblox's Ray Summit talk

[kevin85421]

[Done: v1.4.0] KubeRay operator emits metrics about cluster startup time and others

• [Feature] Emit metrics of cluster creation and other related metrics for observability #2681
• Example: [Feature] Emit metrics of cluster creation and other related metrics for observability #2681 (comment)