Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: HTTPS LoadBalancer Service Not Created #1684

Open
cbl opened this issue Apr 3, 2025 · 0 comments
Open

[Bug]: HTTPS LoadBalancer Service Not Created #1684

cbl opened this issue Apr 3, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@cbl
Copy link

cbl commented Apr 3, 2025

Description

First of all, thank you for your work, this project is truly amazing, it helps a lot and saves plenty of time!

The kube-hetzner module sets up a default LoadBalancer and ships cert-manager by default even though hetzner does have managed certificates. When deploying a kubernetes service via terraform the LoadBalancer has only a service for http on 80. A service for https for port 443 is missing and I would expect it to be created, such that https://foo.com can be reached via the LoadBalancer IP.

LoadBalancer Services: tcp:80 -> 30127

Aparently changing switching the alorithm_type from from round robin to least connection or vice versa leads to the SSL service being created. However as soon as a new kubernetes service is deployed, the LoadBalancer Service on port 443 is removed again.

I wonder what is responsible for creating/deleting the services. Also Hetzner has managed certificates which can be attached to a LoadBalancer directly, what is the purpose in using cert-manager?

Steps to reproduce:

  1. Use the following kube.tf config
  2. Use the following config including:
    • ClusterIsser
    • Deployment (port: 3333)
    • Service (port: 80, target_port: 3333, type: LoadBalancer)
    • Certificate
    • IngressV1 (port: 80)
resource "kubernetes_manifest" "letsencrypt_issuer" {
    manifest = {
        apiVersion = "cert-manager.io/v1"
        kind       = "ClusterIssuer"

        metadata = {
            name = "letsencrypt-prod"
        }

        spec = {
            acme = {
                server = "https://acme-v02.api.letsencrypt.org/directory"
                email  = var.ssl_config.email
                
                privateKeySecretRef = {
                    name = "letsencrypt-prod"
                }

                solvers = [{
                    http01 = {
                        ingress = {
                            class = "nginx"
                        }
                    }
                }]
            }
        }
    }
}
resource "kubernetes_service" "web" {
    metadata {
        name = "web"
        annotations = {
            "load-balancer.hetzner.cloud/name"      = "cluster-nginx"
            "load-balancer.hetzner.cloud/protocol"  = "https"
        }
    }

    spec {
        selector = {
            app = "web"
        }

        port {
            port = 80
            target_port = 3333
        }

        type = "LoadBalancer"
    }
}
resource "kubernetes_manifest" "web_certificate" {
    manifest = {
        apiVersion = "cert-manager.io/v1"
        kind       = "Certificate"

        metadata = {
            name      = "web-cert"
            namespace = "default"
        }

        spec = {
            secretName = "tls"

            issuerRef = {
                name = "letsencrypt-prod"
                kind = "ClusterIssuer"
            }

            dnsNames = [var.domains.web]
        }
    }

    depends_on = [
        kubernetes_manifest.letsencrypt_issuer
    ]
}
resource "kubernetes_ingress_v1" "mastodon_ingress" {
    metadata {
        name = "web"
        annotations = {
            "kubernetes.io/ingress.class" = "nginx"
        }
    }
    spec {
        tls {
            hosts       = [var.domain]
            secret_name = "tls"
        }
        rule {
            host = var.domain
            http {
                path {
                    path      = "/"
                    path_type = "Prefix"
                    backend {
                        service {
                            name = "web"
                            port {
                                number = 80
                            }
                        }
                    }
                }
            }
        }
    }
}

Kube.tf file

module "kube-hetzner" {
    providers = {
        hcloud = hcloud
    }

    hcloud_token = var.hcloud_token
    source = "kube-hetzner/kube-hetzner/hcloud"
    ssh_public_key = file("~/.ssh/id_ed25519.pub")
    ssh_private_key = file("~/.ssh/id_ed25519")

    network_region = "eu-central"
    cluster_name = var.cluster_name

    control_plane_nodepools = [
        {
            name        = "control-plane-nbg1",
            server_type = "cx22",
            location    = "nbg1",
            labels      = [],
            taints      = [],
            count       = 1
        }
    ]

    agent_nodepools = [
        {
            name        = "agent-web-nbg1",
            server_type = "cx22",
            location    = "nbg1",
            labels      = ["node-type=web"],
            taints      = [],
            count       = 1
        },
    ]

    load_balancer_location  = "nbg1"
    load_balancer_type      = "lb11"
    ingress_controller = "nginx"
    create_kubeconfig = true
}

Logs

Warning   
SyncLoadBalancerFailed   
service/foo-web  
Error syncing load balancer: failed to ensure load balancer: hcloud/loadBalancers.EnsureLoadBalancer: hcops/LoadBalancerOps.ReconcileHCLBServices: hcops/hclbServiceOptsBuilder.buildUpdateServiceOpts: hcops/hclbServiceOptsBuilder.extract: hcops/hclbServiceOptsBuilder.resolveCertsByNameOrID: hcops/CertificateOps.GetCertificateByNameOrID: not found

Screenshots

No response

Platform

Fedora release 41 (Forty One)

@cbl cbl added the bug Something isn't working label Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant