Skip to content

knative-serving: Workaround Istio race wrt Gateway creation #2092

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

kimwnasptd
Copy link
Member

Which issue is resolved by this Pull Request:
Resolves #2082

Description of your changes:
When deploying knative-serving there a race due to an Istio issue that ends up having cluster-local-gateway misconfigured because of conflicting ports. In a nutshell Istio creates listeners based on Gateway ports and the corresponding Services (if any) and in our case we have two Services (cluster-local-gateway, knative-local-gateway) with port 80 and different targetPort (8080, 8081). In normal operation the pod gets the following listeners:

ADDRESS PORT  MATCH DESTINATION
0.0.0.0 8080  ALL   Route: http.80
0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and istio complaining with:

Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that it gets ignored by istio when translating gateway ports.

/cc @elikatsis

When deploying knative-serving there a race due to an Istio issue
that ends up having cluster-local-gateway misconfigured because of
conflicting ports. In a nutshell Istio creates listeners based on
Gateway ports and the corresponding Services (if any) and in our
case we have two Services (cluster-local-gateway, knative-local-gateway)
with port 80 and different targetPort (8080, 8081). In normal
operation the pod gets the following listeners:

  ADDRESS PORT  MATCH DESTINATION
  0.0.0.0 8080  ALL   Route: http.80
  0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and
istio complaining with:

  Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that
it gets ignored by istio when translating gateway ports.

Signed-off-by: Kimonas Sotirchos <[email protected]>
@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kimwnasptd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@elikatsis
Copy link
Member

Thanks @kimwnasptd !

/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Dec 22, 2021
@google-oss-prow google-oss-prow bot merged commit b9526c9 into kubeflow:master Dec 22, 2021
kimwnasptd added a commit to arrikto/kubeflow-manifests that referenced this pull request Dec 22, 2021
…#2092)

When deploying knative-serving there a race due to an Istio issue
that ends up having cluster-local-gateway misconfigured because of
conflicting ports. In a nutshell Istio creates listeners based on
Gateway ports and the corresponding Services (if any) and in our
case we have two Services (cluster-local-gateway, knative-local-gateway)
with port 80 and different targetPort (8080, 8081). In normal
operation the pod gets the following listeners:

  ADDRESS PORT  MATCH DESTINATION
  0.0.0.0 8080  ALL   Route: http.80
  0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and
istio complaining with:

  Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that
it gets ignored by istio when translating gateway ports.

Signed-off-by: Kimonas Sotirchos <[email protected]>
google-oss-prow bot pushed a commit that referenced this pull request Dec 22, 2021
…2094)

When deploying knative-serving there a race due to an Istio issue
that ends up having cluster-local-gateway misconfigured because of
conflicting ports. In a nutshell Istio creates listeners based on
Gateway ports and the corresponding Services (if any) and in our
case we have two Services (cluster-local-gateway, knative-local-gateway)
with port 80 and different targetPort (8080, 8081). In normal
operation the pod gets the following listeners:

  ADDRESS PORT  MATCH DESTINATION
  0.0.0.0 8080  ALL   Route: http.80
  0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and
istio complaining with:

  Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that
it gets ignored by istio when translating gateway ports.

Signed-off-by: Kimonas Sotirchos <[email protected]>
yhwang pushed a commit to yhwang/manifests that referenced this pull request Jan 4, 2022
…#2092)

When deploying knative-serving there a race due to an Istio issue
that ends up having cluster-local-gateway misconfigured because of
conflicting ports. In a nutshell Istio creates listeners based on
Gateway ports and the corresponding Services (if any) and in our
case we have two Services (cluster-local-gateway, knative-local-gateway)
with port 80 and different targetPort (8080, 8081). In normal
operation the pod gets the following listeners:

  ADDRESS PORT  MATCH DESTINATION
  0.0.0.0 8080  ALL   Route: http.80
  0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and
istio complaining with:

  Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that
it gets ignored by istio when translating gateway ports.

Signed-off-by: Kimonas Sotirchos <[email protected]>
Tomcli pushed a commit to IBM/manifests that referenced this pull request Jan 4, 2022
…#2092)

When deploying knative-serving there a race due to an Istio issue
that ends up having cluster-local-gateway misconfigured because of
conflicting ports. In a nutshell Istio creates listeners based on
Gateway ports and the corresponding Services (if any) and in our
case we have two Services (cluster-local-gateway, knative-local-gateway)
with port 80 and different targetPort (8080, 8081). In normal
operation the pod gets the following listeners:

  ADDRESS PORT  MATCH DESTINATION
  0.0.0.0 8080  ALL   Route: http.80
  0.0.0.0 8081  ALL   Route: http.8081

We have seen on fresh deployment to end up with only 8081:8081 and
istio complaining with:

  Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

To fix that add a label in knative-local-gateway service so that
it gets ignored by istio when translating gateway ports.

Signed-off-by: Kimonas Sotirchos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Istio and Knative problematic Gateway ports mapping in KF 1.4
2 participants