Skip to content

prewarm runtime pods are not completely deleted #652

Closed
@ningyougang

Description

@ningyougang

If the number of prewarm runtime pods is small, after executed helm delete(or uninstall) xxx, all prewarm runtime pods are deleted completely.

But if the number of prewarm runtime pods is big, after executed helm delete(or uninstall) xxx, some prewarm runtime pods are not deleted completely.(e.g. maybe 1/3 prewarm pods are deleted, 2/3 prewarm pods are not deleted, still Running)

The reason is
After analized the kubernets-client-4.4.2.jar source codes, when invoke below codes

        kubeRestClient
          .inNamespace(kubeRestClient.getNamespace)
          .pods()
          .withLabels(labels.asJava)
          .delete()

Finally, it will invoke below codes
image
We can see, it deletes the runtime pods in a for statement, on the other hand, h.delete(client, ...) method is a sync method, due to some configuration of timeout is low, it leads to can't delete all runtime pods(When timeout happens, above for statement is not executed finished)

I will open 2 prs to solve this issue

if increase the timeout value, can delete all runtime pods successfully, e.g.
values.yaml

invoker:
  runtimeDeleteTimeout: "5 minutes"
akka:
  actorSystemTerminateTimeout: "5 m"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions