-
Notifications
You must be signed in to change notification settings - Fork 86
ingress-controller: update ingresses only if the load balancer is active #745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ingress-controller: update ingresses only if the load balancer is active #745
Conversation
Overall approach to check loadbalancer status looks OK. |
made the changes to address this |
TBD
|
de27916
to
152337a
Compare
type ELBV2API interface { | ||
elbv2.DescribeTargetGroupsAPIClient | ||
elbv2.DescribeTargetHealthAPIClient | ||
elbv2.DescribeLoadBalancersAPIClient |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this likely needs also an adjustment of our documentation about what AWS policy you need, and if so we would need also to change minor version in VERSION file and communicate the change in the release.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes introduced here need permission for elasticloadbalancing:DescribeLoadBalancers
and cloudformation:ListStackReources
Actions. The requirements.md already have them documents as follows.
{
"Version": "2012-10-17",
"Statement": [
...
{
"Action": "elasticloadbalancing:DescribeLoadBalancers",
"Resource": "*",
"Effect": "Allow"
},
...
{
"Action": "cloudformation:List*",
"Resource": "*",
"Effect": "Allow"
},
...
]
}
8e8a668
to
e0596e6
Compare
As per the existing behavior, once the stack update completes, the ingress controller immediately updates the status of all ingresses and routegroups to reference the new load balancer. This update may sometimes happens before the new load balancer has marked its targets (skipper-ingress) as healthy, leading clients being routed to a load balancer that is not ready to serve traffic yet. To improve this behavior we retrieve the ELBs via aws api and build the models including the ELB state of the corresponding ELB. Then before updating the ingresses/routegroups (updateIngress func) we check whether the ELB is in active state, if not we skip updating the ingresses/routegroups status with the ELB hostname. Signed-off-by: Thilina Madumal <[email protected]>
Refactor fake certificate provider to enable creation of multiple certificates with the same CA and to allow manual mocking of GetCertificates result. Follow up on zalando-incubator#615 and zalando-incubator#587 Signed-off-by: Alexander Yastrebov <[email protected]>
Controller manages loadbalancer lifecycle based on available certificates. Prevent controller from deleting loadbalancers when there are no certificates found e.g. due to a bug like zalando-incubator#725 or a tag filter misconfiguration (zalando-incubator#658). Signed-off-by: Alexander Yastrebov <[email protected]>
e0596e6
to
2169f9c
Compare
Closing this PR in favour of #748. I messed the commit history while rebasing, it was easier to cherrypick the changes and create a new PR rather than trying to fix the commit history I messed up here. |
Issue Description
As per the existing behavior, once the stack update completes, the
ingress controller immediately updates the status of all ingresses and
routegroups to reference the new load balancer. This update may
sometimes happens before the new load balancer has marked its targets
(skipper-ingress) as healthy, leading clients being routed to a load
balancer that is not ready to serve traffic yet.
Fix
To improve this behavior we retrieve the ELBs via aws api and build
the models including the ELB state of the corresponding ELB. Then
before updating the ingresses/routegroups (updateIngress func) we
check whether the ELB is in active state, if not we skip updating
the ingresses/routegroups status with the ELB hostname.