|
| 1 | +# Istio |
| 2 | + |
| 3 | +## Upgrade Istio Manifests |
| 4 | + |
| 5 | +Istio ships with an installer called `istioctl`, which is a deployment / |
| 6 | +debugging / configuration management tool for Istio all in one package. |
| 7 | +In this section, we explain how to upgrade our istio kustomize packages |
| 8 | +by leveraging `istioctl`. Assuming the new version is `X.Y.Z` and the |
| 9 | +old version is `X1.Y1.Z1`: |
| 10 | + |
| 11 | +1. Make a copy of the old istio manifests tree, which will become the |
| 12 | + kustomization for the new Istio version: |
| 13 | + |
| 14 | + $ export MANIFESTS_SRC=<path/to/manifests/repo> |
| 15 | + $ export ISTIO_OLD=$MANIFESTS_SRC/common/istio-X1-Y1 |
| 16 | + $ export ISTIO_NEW=$MANIFESTS_SRC/common/istio-X-Y |
| 17 | + $ cp -a $ISTIO_OLD $ISTIO_NEW |
| 18 | + |
| 19 | +2. Download `istioctl` for version `X.Y.Z`: |
| 20 | + |
| 21 | + $ ISTIO_VERSION="X.Y.Z" |
| 22 | + $ wget "https://github.com/istio/istio/releases/download/${ISTIO_VERSION}/istio-${ISTIO_VERSION}-linux.tar.gz" |
| 23 | + $ tar xvfz istio-${ISTIO_VERSION}-linux.tar.gz |
| 24 | + # sudo mv istio-${ISTIO_VERSION}/bin/istioctl /usr/local/bin/istioctl |
| 25 | + |
| 26 | +3. Use `istioctl` to generate an `IstioOperator` resource, the |
| 27 | + CustomResource used to describe the Istio Control Plane: |
| 28 | + |
| 29 | + $ cd $ISTIO_NEW |
| 30 | + $ istioctl profile dump demo > profile.yaml |
| 31 | + |
| 32 | + --- |
| 33 | + **NOTE** |
| 34 | + |
| 35 | + `istioctl` comes with a bunch of [predefined |
| 36 | + profiles](https://istio.io/v1.9/docs/setup/additional-setup/config-profiles/) |
| 37 | + (`default`, `demo`, `minimal`, etc.). The `default` profile is installed by default. |
| 38 | + |
| 39 | + --- |
| 40 | + |
| 41 | +4. Generate manifests and add them to their respective packages. We |
| 42 | + will generate manifests using `istioctl`, the |
| 43 | + `profile.yaml` file from upstream and the |
| 44 | + `profile-overlay.yaml` file that contains our desired |
| 45 | + changes: |
| 46 | + |
| 47 | + $ export PATH="$MANIFESTS_SRC/scripts:$PATH" |
| 48 | + $ cd $ISTIO_NEW |
| 49 | + $ istioctl manifest generate --cluster-specific -f profile.yaml -f profile-overlay.yaml > dump.yaml |
| 50 | + $ split-istio-packages -f dump.yaml |
| 51 | + $ mv $ISTIO_NEW/crd.yaml $ISTIO_NEW/istio-crds/base |
| 52 | + $ mv $ISTIO_NEW/install.yaml $ISTIO_NEW/istio-install/base |
| 53 | + $ mv $ISTIO_NEW/cluster-local-gateway.yaml $ISTIO_NEW/cluster-local-gateway/base |
| 54 | + |
| 55 | + --- |
| 56 | + **NOTE** |
| 57 | + |
| 58 | + `split-istio-packages` is a python script in the same folder as this file. |
| 59 | + The `ruamel.yaml` version used is 0.16.12. |
| 60 | + |
| 61 | + `--cluster-specific` is a flag that determines if a current K8s cluster context will be used to dynamically |
| 62 | + detect default settings. Ensure you have a target cluster ready before running the above commands. |
| 63 | + We set this flag because `istioctl manifest generate` generates manifest files with resources that are no |
| 64 | + longer supported in Kubernetes 1.25 (`policy/v1beta1`). See: https://github.com/istio/istio/issues/41220 |
| 65 | + |
| 66 | + --- |
| 67 | + |
| 68 | +5. Remove PodDisruptionBudget from `istio-install` and `cluster-local-gateway` kustomizations. |
| 69 | + See https://github.com/istio/istio/issues/12602 and https://github.com/istio/istio/issues/24000 |
| 70 | + |
| 71 | + Until now we have used two patches: |
| 72 | + - `common/istio-1-16/istio-install/base/patches/remove-pdb.yaml` |
| 73 | + - `common/istio-1-16/cluster-local-gateway/base/patches/remove-pdb.yaml` |
| 74 | + |
| 75 | + The above patches do not work with kustomize v3.2.0 as it doesn't have the appropriate |
| 76 | + openapi schemas for the policy/v1 API version resources. This is fixed in kustomize v4+. |
| 77 | + See https://github.com/kubernetes-sigs/kustomize/issues/3694#issuecomment-799700607 and |
| 78 | + https://github.com/kubernetes-sigs/kustomize/issues/4495 |
| 79 | + |
| 80 | + A temporary workaround is to use the following instructions to manually delete the PodDisruptionBudget resources with `yq`: |
| 81 | + |
| 82 | + $ yq eval -i 'select((.kind == "PodDisruptionBudget" and .metadata.name == "cluster-local-gateway") | not)' common/istio-1-16/cluster-local-gateway/base/cluster-local-gateway.yaml |
| 83 | + $ yq eval -i 'select((.kind == "PodDisruptionBudget" and .metadata.name == "istio-ingressgateway") | not)' common/istio-1-16/istio-install/base/install.yaml |
| 84 | + $ yq eval -i 'select((.kind == "PodDisruptionBudget" and .metadata.name == "istiod") | not)' common/istio-1-16/istio-install/base/install.yaml |
| 85 | + |
| 86 | + --- |
| 87 | + **NOTE** |
| 88 | + |
| 89 | + NOTE: Make sure to remove a redundant {} at the end of the `common/istio-1-16/istio-install/base/install.yaml` and `common/istio-1-16/cluster-local-gateway/base/cluster-local-gateway.yaml` files. |
| 90 | + |
| 91 | + --- |
| 92 | + |
| 93 | +## Changes to Istio's upstream manifests |
| 94 | + |
| 95 | +### Changes to the upstream IstioOperator profile |
| 96 | + |
| 97 | +Changes to Istio's upstream profile `demo` are the following: |
| 98 | + |
| 99 | +- Add a `cluster-local-gateway` component for KFServing. |
| 100 | +- Disable the EgressGateway component. We don\'t use it and it adds |
| 101 | + unnecessary complexity. |
| 102 | + |
| 103 | +Those changes are captured in the [profile-overlay.yaml](profile-overlay.yaml) |
| 104 | +file. |
| 105 | + |
| 106 | +### Changes to the upstream manifests using kustomize |
| 107 | + |
| 108 | +The Istio kustomizations make the following changes: |
| 109 | + |
| 110 | +- Remove PodDisruptionBudget from `istio-install` and `cluster-local-gateway` kustomizations. See: |
| 111 | + - https://github.com/istio/istio/issues/12602 |
| 112 | + - https://github.com/istio/istio/issues/24000 |
| 113 | +- Add EnvoyFilter for adding an `X-Forwarded-For` header in requests passing through the Istio Ingressgateway, inside the `istio-install` kustomization. |
| 114 | +- Add Istio AuthorizationPolicy to allow all requests to the Istio Ingressgateway and the Istio cluster-local gateway. |
| 115 | +- Add Istio AuthorizationPolicy in Istio's root namespace, so that sidecars deny traffic by default (explicit deny-by-default authorization model). |
| 116 | +- Add Gateway CRs for the Istio Ingressgateway and the Istio cluster-local gateway, as `istioctl` stopped generating them in later versions. |
| 117 | +- Add the istio-system namespace object to `istio-namespace`, as `istioctl` stopped generating it in later versions. |
| 118 | +- Configure TCP KeepAlives. |
| 119 | +- Disable tracing as it causes DNS breakdown. See: |
| 120 | + https://github.com/istio/istio/issues/29898 |
0 commit comments