kube-addon-operator keeps crashing

[jlebon]

Let me know if this isn't the right place for component issues.

Versions

# rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://rhcos:openshift/3.10/x86_64/os
                   Version: 4.0.4357 (2018-08-15 18:29:40)
                    Commit: 75574ee3d0b788386b6fe715b2179b0425f98c46ea21724b1967be16b8faee4a
# rpm -qa | grep ^origin
origin-node-3.11.0-0.alpha.0.893.80abd58.x86_64
origin-clients-3.11.0-0.alpha.0.893.80abd58.x86_64
origin-hyperkube-3.11.0-0.alpha.0.893.80abd58.x86_64

Terraform version (terraform version):

$ terraform version
Terraform v0.11.7

Your version of Terraform is out of date! The latest version
is 0.11.8. You can update by downloading from www.terraform.io/downloads.html

Platform (aws|azure|openstack|metal|vmware):

libvirt

What happened?

The kube-addon-operator keeps crashing:

$ oc get -n tectonic-system pods
NAME                                         READY     STATUS    RESTARTS   AGE
directory-sync-d84d84d9f-frfgs               1/1       Running   0          2h
kube-addon-operator-6bdf5dcd67-zwgkb         1/1       Running   14         2h
tectonic-alm-operator-79b6996f74-rd77l       1/1       Running   0          2h
tectonic-channel-operator-f545c8db8-lt5kr    1/1       Running   0          2h
tectonic-clu-6b8d87785f-pnx9g                1/1       Running   0          2h
tectonic-node-agent-7cqxf                    1/1       Running   2          2h
tectonic-node-agent-tts6j                    1/1       Running   3          2h
tectonic-stats-emitter-d87f669fd-mxldv       2/2       Running   0          2h
tectonic-utility-operator-6c7b696f79-6mhlv   1/1       Running   0          2h
$ oc logs -n tectonic-system kube-addon-operator-6bdf5dcd67-zwgkb
I0823 19:25:00.088020       1 run.go:60] kube-addon-operator starting
I0823 19:25:00.287656       1 leaderelection.go:174] attempting to acquire leader lease...
E0823 19:25:00.493659       1 event.go:260] Could not construct reference to: '&v1.ConfigMap{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-addon", GenerateName:"", Namespace:"tectonic-system", SelfLink:"
/api/v1/namespaces/tectonic-system/configmaps/kube-addon", UID:"5dd8de70-a6f4-11e8-8854-be841c320f7d", ResourceVersion:"24966", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63670639711, loc:(*time.Location)(0x197566
0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"control-plane.alpha.kubernetes.io/leader":"{\"holderIdentity\":\"kube-addon-operator-6bdf5dcd6
7-zwgkb\",\"leaseDurationSeconds\":90,\"acquireTime\":\"2018-08-23T16:48:30Z\",\"renewTime\":\"2018-08-23T19:25:00Z\",\"leaderTransitions\":0}"}, OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]
string(nil), ClusterName:""}, Data:map[string]string(nil)}' due to: 'no kind is registered for the type v1.ConfigMap'. Will not report event: 'Normal' 'LeaderElection' 'kube-addon-operator-6bdf5dcd67-zwgkb became leader'
I0823 19:25:00.493747       1 leaderelection.go:184] successfully acquired lease tectonic-system/kube-addon
I0823 19:25:00.493781       1 run.go:99] started leading: running kube-addon-operator
I0823 19:25:01.587697       1 update.go:53] Upgrade triggered, req: 1, comp: 0
I0823 19:25:24.187116       1 update.go:78] Start upgrading
I0823 19:25:25.322595       1 component.go:79] Updating component ClusterRoleBinding/metrics-server:system:auth-delegator
I0823 19:25:26.089057       1 component.go:101] Finished update of component: ClusterRoleBinding/metrics-server:system:auth-delegator
I0823 19:25:27.096713       1 component.go:79] Updating component RoleBinding/kube-system/metrics-server-auth-reader
I0823 19:25:27.454396       1 component.go:101] Finished update of component: RoleBinding/kube-system/metrics-server-auth-reader
I0823 19:25:28.462451       1 component.go:79] Updating component APIService/v1beta1.metrics.k8s.io
I0823 19:25:28.789396       1 component.go:101] Finished update of component: APIService/v1beta1.metrics.k8s.io
I0823 19:25:29.888677       1 component.go:79] Updating component Deployment/kube-system/metrics-server
I0823 19:25:31.387669       1 component.go:101] Finished update of component: Deployment/kube-system/metrics-server
I0823 19:25:32.397426       1 component.go:79] Updating component Service/kube-system/metrics-server
I0823 19:25:32.787819       1 component.go:101] Finished update of component: Service/kube-system/metrics-server
I0823 19:25:33.791862       1 component.go:79] Updating component ServiceAccount/kube-system/metrics-server
I0823 19:25:34.088068       1 component.go:101] Finished update of component: ServiceAccount/kube-system/metrics-server
I0823 19:25:35.095802       1 component.go:79] Updating component ClusterRole/system:metrics-server
I0823 19:25:35.399154       1 component.go:101] Finished update of component: ClusterRole/system:metrics-server
I0823 19:25:36.413012       1 component.go:79] Updating component ClusterRoleBinding/system:metrics-server
I0823 19:25:36.636727       1 component.go:101] Finished update of component: ClusterRoleBinding/system:metrics-server
I0823 19:25:37.665680       1 component.go:79] Updating component ConfigMap/openshift-web-console/webconsole-config
I0823 19:25:37.908471       1 component.go:101] Finished update of component: ConfigMap/openshift-web-console/webconsole-config
I0823 19:25:38.917026       1 component.go:79] Updating component ServiceAccount/openshift-web-console/webconsole
I0823 19:25:39.192305       1 component.go:101] Finished update of component: ServiceAccount/openshift-web-console/webconsole
I0823 19:25:40.200972       1 component.go:79] Updating component Service/openshift-web-console/webconsole
I0823 19:25:40.401035       1 component.go:101] Finished update of component: Service/openshift-web-console/webconsole
I0823 19:25:41.409767       1 component.go:79] Updating component Deployment/openshift-web-console/webconsole
I0823 19:25:43.091352       1 component.go:101] Finished update of component: Deployment/openshift-web-console/webconsole
I0823 19:25:44.134619       1 component.go:79] Updating component ClusterRoleBinding/registry-registry-role
I0823 19:25:44.414042       1 component.go:101] Finished update of component: ClusterRoleBinding/registry-registry-role
I0823 19:25:45.421271       1 component.go:79] Updating component ServiceAccount/default/registry
I0823 19:25:45.689676       1 component.go:101] Finished update of component: ServiceAccount/default/registry
I0823 19:25:46.696835       1 update.go:121] Not current object default/docker-registry is running
I0823 19:25:46.696853       1 component.go:196] Install mode, no original object is needed, skipping.
I0823 19:25:46.697898       1 component.go:226] No original or current manifest found, it's in install mode for manifest for default/docker-registry
I0823 19:25:46.697914       1 component.go:79] Updating component Service/default/docker-registry
E0823 19:25:46.936398       1 update.go:66] Error updating: Failed update of component: Service/default/docker-registry due to: Service "docker-registry" is invalid: spec.clusterIP: Invalid value: "10.3.0.25": provided IP is not in the val
id range. The range of valid IPs is 172.18.0.0/16
I0823 19:25:57.945730       1 update.go:53] Upgrade triggered, req: 1, comp: 0
I0823 19:26:11.187993       1 update.go:78] Start upgrading
...

What you expected to happen?

The kube-addon-operator not crashing.

How to reproduce it (as minimally and precisely as possible)?

Run installer on top of RHCOS. Also pulled in #134 and #150.

[abhinavdahiya]

kube-addon-operator currently only supports 10.30.0.0/16 cidr range for services....
registry operator will fix it in future..

[abhinavdahiya]

/close

[openshift-ci-robot]

@abhinavdahiya: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.