After install on Ubuntu 22.04, microk8s is not running.

[gerryseidlEpiq]

Summary

After install, microk8s is not running.
Running - microk8s kubectl get all --all-namespaces shows that components are not running.
I tried to stop and start, and also reboot. No change.

Introspection Report

inspection-report-20231011_172501.tar.gz

[gerryseidlEpiq]

Does anyone have any advice on this issue??

[ktsakalozos]

Hi @gerryseidlEpiq

I see that the K8s services have started but are failing to fetch container images. If you do a microk8s kubectl get po -A you should see:

NAMESPACE     NAME                                             READY   STATUS     RESTARTS   AGE   IP            NODE                          NOMINATED NODE   READINESS GATES
kube-system   pod/calico-node-5hww5                            0/1     Init:0/2   0          46m   10.92.13.25   z077ocrmsfr02.dqscust.local   <none>           <none>
kube-system   pod/calico-kube-controllers-6c99c8747f-gzkv4     0/1     Pending    0          46m   <none>        <none>                        <none>           <none>
kube-system   pod/coredns-7745f9f87f-mb9nk                     0/1     Pending    0          46m   <none>        <none>                        <none>           <none>
kube-system   pod/metrics-server-7747f8d66b-ndclv              0/1     Pending    0          45m   <none>        <none>                        <none>           <none>
kube-system   pod/kubernetes-dashboard-fc86bcc89-tghlg         0/1     Pending    0          45m   <none>        <none>                        <none>           <none>
kube-system   pod/dashboard-metrics-scraper-5cb4f4bb9c-pvqrp   0/1     Pending    0          45m   <none>        <none>                        <none>           <none>
kube-system   pod/hostpath-provisioner-58694c9f4b-8jpqz        0/1     Pending    0          45m   <none>        <none>                        <none>           <none>

If you now do a microk8s kubectl describe -n kube-system pod/calico-node-5hww5 you should see that there is a problem fetching the pause container. The containerd error is:

Oct 11 17:24:22 z077ocrmsfr02.dqscust.local microk8s.daemon-containerd[3927662]: time="2023-10-11T17:24:22.578745151Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-5hww5,Uid:4f417c97-a26a-4490-81fa-d0fe26669e62,Namespace:kube-system,Attempt:0,} failed, error" error="failed to get sandbox image \"registry.k8s.io/pause:3.7\": failed to pull image \"registry.k8s.io/pause:3.7\": failed to pull and unpack image \"registry.k8s.io/pause:3.7\": failed to resolve reference \"registry.k8s.io/pause:3.7\": failed to do request: Head \"https://registry.k8s.io/v2/pause/manifests/3.7\": net/http: TLS handshake timeout"

There is probably something blocking the connection to registry.k8s.io. Are you behind a proxy or a firewall?

[Bruno-Dompro]

Hi,

Just new in This world...
I installed new Ubuntu server (22.0.4 LTS) , snapd and microk8s.
But I'm stuck on "microk8s is not running. Use microk8s inspect for a deeper inspection."
I don't know what to do with the tarball generated.

Someone to have a look and tell me what is going wrong ?

Some Complements:

~$ snap list
Name Version Rev Tracking Publisher Notes
core20 20240111 2182 latest/stable canonical✓ base
lxd 5.0.3-babaaf8 27948 5.0/stable/… canonical✓ -
microk8s v1.29.2 6641 1.29/stable canonical✓ classic
snapd 2.61.2 21184 latest/stable canonical✓ snapd
~$

Thanks in advance.

inspection-report-20240409_115121.tar.gz

[neoaggelos]

Hi @Bruno-Dompro I see the following line repeating on the containerd logs. Looks like you are having trouble reaching https://registry.k8s.io?

Apr 09 10:10:28 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:28.375105462Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-4d9v6,Uid:b1c208fb-3c75-4133-aca4-a977f847d2f5,Namespace:kube-system,Attempt:0,} failed, error" error="failed to get sandbox image \"registry.k8s.io/pause:3.7\": failed to pull image \"registry.k8s.io/pause:3.7\": failed to pull and unpack image \"registry.k8s.io/pause:3.7\": failed to resolve reference \"registry.k8s.io/pause:3.7\": failed to do request: Head \"https://registry.k8s.io/v2/pause/manifests/3.7\": read tcp 10.236.66.244:48096->34.96.108.209:443: read: connection reset by peer"
Apr 09 10:10:39 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:39.327548817Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-4d9v6,Uid:b1c208fb-3c75-4133-aca4-a977f847d2f5,Namespace:kube-system,Attempt:0,}"
Apr 09 10:10:39 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:39.365134532Z" level=info msg="trying next host" error="failed to do request: Head \"https://registry.k8s.io/v2/pause/manifests/3.7\": read tcp 10.236.66.244:51680->34.96.108.209:443: read: connection reset by peer" host=registry.k8s.io

[Bruno-Dompro]

Hi Angelos, I suspected something like that from our FW PaloAlto... But I didn't found directly this information from the Tarball files. I'll ask my Security Admin to open the access to "**.k8s.io <http://k8s.io>* ". Overall, are they other sites needed to deploy correctly the microk8s pack ? Many Thanks for your answer. Regards Le jeu. 11 avr. 2024 à 18:40, Angelos Kolaitis ***@***.***> a écrit :

…

Hi @Bruno-Dompro <https://github.com/Bruno-Dompro> I see the following line repeating on the containerd logs. Looks like you are having trouble reaching https://registry.k8s.io? Apr 09 10:10:28 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:28.375105462Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-4d9v6,Uid:b1c208fb-3c75-4133-aca4-a977f847d2f5,Namespace:kube-system,Attempt:0,} failed, error" error="failed to get sandbox image \"registry.k8s.io/pause:3.7\ <http://registry.k8s.io/pause:3.7%5C>": failed to pull image \"registry.k8s.io/pause:3.7\ <http://registry.k8s.io/pause:3.7%5C>": failed to pull and unpack image \"registry.k8s.io/pause:3.7\ <http://registry.k8s.io/pause:3.7%5C>": failed to resolve reference \"registry.k8s.io/pause:3.7\ <http://registry.k8s.io/pause:3.7%5C>": failed to do request: Head \"https://registry.k8s.io/v2/pause/manifests/3.7\": read tcp 10.236.66.244:48096->34.96.108.209:443: read: connection reset by peer" Apr 09 10:10:39 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:39.327548817Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-4d9v6,Uid:b1c208fb-3c75-4133-aca4-a977f847d2f5,Namespace:kube-system,Attempt:0,}" Apr 09 10:10:39 pfrdpmagicimm microk8s.daemon-containerd[878]: time="2024-04-09T10:10:39.365134532Z" level=info msg="trying next host" error="failed to do request: Head \"https://registry.k8s.io/v2/pause/manifests/3.7\": read tcp 10.236.66.244:51680->34.96.108.209:443: read: connection reset by peer" host=registry.k8s.io — Reply to this email directly, view it on GitHub <#4248 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHWMU3E4DMOB3ZGJCTYQHX3Y424IBAVCNFSM6AAAAAA54P3TC2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJQGA4TGOBQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Bruno-Dompro]

Pb solved by modifying WF rules to let outgoing.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.