Skip to content

Commit 1b9e0d6

Browse files
Jeffwank8s-ci-robot
authored andcommitted
Merge installation guidance for new and existing cluster (#1072)
* Merge installation guidance for new and existing cluster * Address code review feedbacks
1 parent 2b5c19a commit 1b9e0d6

File tree

4 files changed

+37
-84
lines changed

4 files changed

+37
-84
lines changed

content/_redirects

+4-1
Original file line numberDiff line numberDiff line change
@@ -91,4 +91,7 @@ docs/started/requirements/ /docs/started/getting-started/
9191
/docs/other-guides/job-scheduling/ /docs/use-cases/job-scheduling/
9292
/docs/other-guides/upgrade/ /docs/upgrading/upgrade/
9393

94-
/docs/use-cases/kubeflow-on-multinode-cluster/ /docs/other-guides/kubeflow-on-multinode-cluster/
94+
/docs/use-cases/kubeflow-on-multinode-cluster/ /docs/other-guides/kubeflow-on-multinode-cluster/
95+
96+
# Remove Kubeflow installation on existing EKS cluster
97+
/docs/aws/deploy/existing-cluster/ /docs/aws/deploy/install-kubeflow/

content/docs/aws/deploy/existing-cluster.md

-63
This file was deleted.

content/docs/aws/deploy/install-kubeflow.md

+31-18
Original file line numberDiff line numberDiff line change
@@ -17,50 +17,64 @@ deploy Kubeflow on Amazon Web Services (AWS).
1717
* Enter your preferred AWS Region and default output options.
1818
* Install [eksctl](https://github.com/weaveworks/eksctl) (version 0.1.31 or newer) and the [aws-iam-authenticator](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html).
1919

20-
You do not need to have an existing Amazon Elastic Container Service for Kubernetes (Amazon EKS) cluster. The deployment process will create a cluster for you.
20+
## EKS cluster
21+
There're many ways to provision EKS cluster, using AWS EKS CLI, CloudFormation or Terraform, AWS CDK or eksctl.
22+
Here, we highly recommend you to create an EKS cluster using [eksctl](https://github.com/weaveworks/eksctl).
23+
24+
You are required to have an existing Amazon Elastic Container Service for Kubernetes (Amazon EKS) cluster before moving the next step.
2125

2226
The installation tool uses the `eksctl` command and doesn't support the `--profile` option in that command.
2327
If you need to switch role, use the `aws sts assume-role` commands. See the AWS guide to [using temporary security credentials to request access to AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html).
2428

2529

2630
## Kubeflow installation
31+
In order to deploy Kubeflow on your existing Amazon EKS cluster, you need to provide `AWS_CLUSTER_NAME`, `cluster region` and `worker roles`.
32+
2733

2834
1. Download the latest `kfctl` golang binary from [Kubeflow release page](https://github.com/kubeflow/kubeflow/releases) and unpack it.
2935

30-
```
36+
```shell
3137
# Add kfctl to PATH, to make the kfctl binary easier to use.
3238
tar -xvf kfctl_<release tag>_<platform>.tar.gz
3339
export PATH=$PATH:"<path to kfctl>"
34-
```
35-
36-
1. Download manifest and config setups
3740

38-
```shell
39-
# Download config files
41+
# Download config file
4042
export CONFIG="/tmp/kfctl_aws.yaml"
4143
wget https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_aws.yaml -O ${CONFIG}
4244
```
4345

4446
* `kfctl_aws.yaml` is one of setup manifests, please check [kfctl_aws_cognito.yaml](https://github.com/kubeflow/kubeflow/blob/master/bootstrap/config/kfctl_aws_cognito.yaml) for the template to enable authentication.
4547

46-
- If you plan to use `kfctl` to create a new eks cluster, please remove follow lines in the manifest file.
47-
- If you want to install on existing EKS cluster, please change roles to your worker node group roles. See [existing cluster](/docs/aws/deploy/existing-cluster) for details.
48+
1. Customize your config file. Retrieve the Amazon EKS cluster name, AWS Region, and IAM role name for your worker nodes.
4849

49-
```shell
50-
roles:
51-
- eksctl-kubeflow-aws-nodegroup-ng-a2-NodeInstanceRole-xxxxxxx
50+
```shell
51+
export AWS_CLUSTER_NAME=<YOUR EKS CLUSTER NAME>
52+
export KFAPP=${AWS_CLUSTER_NAME}
5253
```
5354

54-
1. Run the following commands to set up your environment and initialize the cluster.
55+
> Note: To get your Amazon EKS worker node IAM role name, you can check IAM setting by running the following commands. This command assumes that you used `eksctl` to create your cluster. If you use other provisioning tools to create your worker node groups, please find the role that is associated with your worker nodes in the Amazon EC2 console.
5556

56-
Since there're many ways to create your cluster, we highly recommend you to get our own eks cluster ready.
57+
```shell
58+
aws iam list-roles \
59+
| jq -r ".Roles[] \
60+
| select(.RoleName \
61+
| startswith(\"eksctl-$AWS_CLUSTER_NAME\") and contains(\"NodeInstanceRole\")) \
62+
.RoleName"
5763
58-
Note: If you would like to install Kubeflow on your existing EKS cluster,
59-
please skip this step and follow the setup instructions for an [existing cluster](/docs/aws/deploy/existing-cluster) instead.
64+
eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6
65+
```
6066

67+
Change cluster region and worker roles names in your `kfctl_aws.yaml`
68+
```yaml
69+
region: us-west-2
70+
roles:
71+
- eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6
6172
```
62-
export KFAPP=kfaws
73+
> If you have multiple node groups, you will see corresponding number of node group roles. In that case, please provide the role names as an array.
6374

75+
1. Run the following commands to set up your environment and initialize the cluster.
76+
77+
```shell
6478
kfctl init ${KFAPP} --config=${CONFIG} -V
6579
cd ${KFAPP}
6680
@@ -71,7 +85,6 @@ If you need to switch role, use the `aws sts assume-role` commands. See the AWS
7185
* KFAPP - Use a relative directory name here rather than absolute path, such as `kfapp`. It will be used as eks cluster name.
7286
* CONFIG - Path to the configuration file
7387

74-
7588
*Important!!!* By default, these scripts create an AWS Application Load Balancer for Kubeflow that is open to public. This is good for development testing and for short term use, but we do not recommend that you use this configuration for production workloads.
7689

7790
To secure your installation, Follow the [instructions](/docs/aws/authentication) to add authentication.

content/docs/aws/deploy/uninstall-kubeflow.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ weight = 10
55
+++
66

77

8-
## Uninstall Kubeflow and delete your Amazon EKS cluster.
8+
## Uninstall Kubeflow
99

1010
```
1111
cd ${KFAPP}
1212
kfctl delete all -V
1313
```
1414

15-
> Note: If you installed Kubeflow on an existing Amazon EKS cluster, these scripts won't tear down your cluster in this step. In this case, you must manually delete your cluster.
15+
> Note: If you installed Kubeflow on an existing Amazon EKS cluster, these scripts won't tear down your cluster in this step. If you want to shutdown EKS cluster, you must manually delete it by yourself.

0 commit comments

Comments
 (0)