(Universal) Idling for DevWorkspace based workspaces

[l0rd]

Is your enhancement related to a problem? Please describe

Workspaces controlled by the che-server are shut down when inactive for a specific period of time (specified by the customCheProperty CHE_LIMITS_WORKSPACE_IDLE_TIMEOUT) but there is no way at the moment to automatically shut down Workspaces controlled by the devworkspace operator.

Describe the solution you'd like

Workspaces idling should work with DevWorkspaces too. A possible solution would be to use a mechanism similar to Knative autoscaling system: expose workspaces activity metrics (e.g. editor requests per second) and at che-server level collect those metrics and decide which DevWorkspaces need to be scaled to zero.

Activity metrics should be exposed by editors (theia, jetbrains, vscode) extensions.

Scale to zero for workspaces without an editor or for workspaces with an editors but without metrics is out of the scope of this epic.

Describe alternatives you've considered

Currently the Web Terminal supports idling using activity ticks sent from the Console to the machine-exec activity rest API. This approach has a few drawbacks: couples machine-exec and the component that tracks the activity, machine-exec requires privileges to scale the dw to zero.

[l0rd]

We had a call this morning where we discussed what's the best Che component to handle that. Here is a summary:

I am against making the DevWorkspace operator responsible for idling workspaces. I would like the DevWorkspace to be as simple as possible and to only be responsible to transforms devfiles into k8s objects. It should not be multi-tenant and should not manage user workspaces (creating, updating, deleting DevWorkspace CRs): those are responsibilities of the server-side components.

@skabashnyuk doesn’t want it to be part of the che-server because he considers that we are not investing in that Java code anymore and the che-server is not cloud native anyway. It should rather be a responsibility of the Che operator that currently manages workspaces namespaces.

@sleshchenko doesn't know what's the ideal solution but delegating idling to an operator may not be the best option: that's not about reconciling custom resources and that's what controllers should do.

@tolusha what's your opinion?

[tolusha]

For me it is strange to delegate this to che-operator. As you said:

hat's not about reconciling custom resources and that's what controllers should do.

[sleshchenko]

Just to share. I looked a bit on the implementation for the referenced https://knative.dev/docs/serving/autoscaling/ and they really have operator-based approach.
As far as I understand:

• they inject proxy, resources monitor and metrics exposure into user's pod;
• they have CRDs for PodAutoscaler and their Metrics (don't know in detail how it's used) https://github.com/knative/serving/blob/main/config/core/300-resources/podautoscaler.yaml#L20 https://github.com/knative/serving/blob/main/config/core/300-resources/metric.yaml#L20;
• they scrape metrics each minute https://github.com/knative/serving/blob/8a0eff7a480b292548c3db7a6f7780cc8c1e5c73/pkg/autoscaler/metrics/collector.go#L301;
• they do reconcile loop which verifies if application should be rescaled or not;

If we go with Server approach, we still need to store somewhere the usage information. The simplest solution without database - configmap, but it may look a bit weird.

[l0rd]

Is using knative PodAutoscaler, and not re-implementing something similar, an option?

[sleshchenko]

As we agreed on today's DevWorkspace Cabal, this issue is not in milestone 3 anymore, but instead, we'll implement (in 3rd Milestone) quick solution on Theia + Machine Exec side #20460.

This issue is relevant and we see the value on stopping by inactivity on Che side, regardless editor that is used.
Our team will do investigation/prepare proposal where exactly this mechanism will live and how flow with work.