Masters should not be excluded from service load balancers

[ljani]

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug

What happened:
I'm running a single node cluster on AWS EC2, ie. scheduling work on the master as well. Now that I tried to add an ELB for my service, the EC2 instance is not associated with the ELB. The ELB gets created but there are 0 EC2 instances associated with it. I'm using ELB for SSL termination, if you wonder why I'd like to load balance a single node cluster.

The service controller logs this message:

I0628 17:54:48.853175       1 event.go:221] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"nginx", UID:"59980933-7afc-11e8-9a8c-0621b993d602", APIVersion:"v1", ResourceVersion:"2052", FieldPath:""}): type: 'Warning' reason: 'UnAvailableLoadBalancer' There are no available nodes for LoadBalancer service default/nginx

What you expected to happen:
The EC2 instance is associated with the ELB.

How to reproduce it (as minimally and precisely as possible):

• Boot an EC2 instance with correct IAM permissions
• Install a single node cluster using kubeadm and cloud-provider=aws
• Untaint the node
• Schedule eg. nginx
• Create an external load balancer for nginx
• View the ELB in AWS console and note the Status: 0 of 0 instances in service

Anything else we need to know?:
The reason for this seems to be the node-role.kubernetes.io/master label. This blocks associating a load balancer with the node. On the other hand this changed what is included, because includeNodeFromNodeList did not check if a node is a master. I'm not sure what would be correct fix. I could try submit a PR, if you guide me how this should behave. Is my scenario even a supported one?

I think this bug should be reproducible on other clouds as well.

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:08:34Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration: AWS
• OS (e.g. from /etc/os-release):

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

• Kernel (e.g. uname -a):

Linux ip-x.eu-central-1.compute.internal 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

• Install tools: kubeadm

[ljani]

#33884 has area/nodecontroller, so I think the correct sig is @kubernetes/sig-node-bugs.

[k8s-ci-robot]

@ljani: Reiterating the mentions to trigger a notification:
@kubernetes/sig-node-bugs

In response to this:

#33884 has area/nodecontroller, so I think the correct sig is @kubernetes/sig-node-bugs.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jhorwit2]

@ljani This is by design, load balancers do not include master nodes in the available pool of backend servers. You should remove the node-role.kubernetes.io/master label from your node and ensure it's schedulable & ready in order for it to be a backend. (exact filter logic lives here)

I'm going to close this issue as it's not a bug. If you would like to see the ability to use masters as available backends for LB's then please create another ticket with a feature request.

/close

[ljani]

@jhorwit2 Thanks for the response. So, it's okay to remove the node-role.kubernetes.io/master label even if there's only one node in the cluster? Are there any drawbacks? I was a bit wary of that, because I thought each cluster would need at least one master and you should only remove the NoSchedule taint from it. The kubeadm guide only refers to removing the taint from the master and not the label itself, but yeah, the guide is for a single master cluster, so it would be a little contradictory.

Searching for masterless does not really yield any related results.

Anyhow, if that's a supported scenario, then I'm very happy with it. Otherwise I should open the ticket.

[jhorwit2]

It all depends on how the cluster is setup and if they depend on the labels or not. Single node clusters will probably always have some quirks.

[ljani]

I've been following the Creating a single master cluster with kubeadm guide and thus I'm using kubeadm. I'll try and see how it goes.

[jhorwit2]

The docs are probably incorrect when it comes to this because it wasn't tested. They should be updated.

/reopen
/remove-sig node
/sig cluster-lifecycle