2
2
3
3
[ ![ Build Status] ( https://travis-ci.com/volcano-sh/volcano.svg?token=sstuqFE81ukmNz9cEEtd&branch=master )] ( https://travis-ci.com/volcano-sh/volcano ) [ ![ slack] ( https://img.shields.io/badge/Volcano-%23SLACK-red.svg )] ( https://volcano-sh.slack.com/messages/CGET876H5/ )
4
4
5
- Volcano is a Kubernetes-based system for high performance workload, providing mechanisms for applications
6
- which would like to run high performance workload leveraging Kubernetes, e.g. Tensorflow, Spark, MPI.
7
-
8
- Volcano builds upon a decade and a half of experience on running high performance workload workloads at scale
9
- using several systems, combined with best-of-breed ideas and practices from the open source community.
5
+ Volcano is system for runnning high performance workloads on
6
+ Kubernetes. It provides a suite of mechanisms currently missing from
7
+ Kubernetes that are commonly required by many classes of high
8
+ performance workload including:
9
+
10
+ 1 . machine learning/deep learning,
11
+ 2 . bioinformatics/genomics, and
12
+ 3 . other "big data" applications.
13
+
14
+ These types of applications typically run on generalized domain
15
+ frameworks like Tensorflow, Spark, PyTorch, MPI, etc, which Volcano integrates with.
16
+
17
+ Some examples of the mechanisms and features that Volcano adds to Kubernetes are:
18
+
19
+ 1 . Job management extensions and improvements, e.g:
20
+ 1 . Multi-pod jobs
21
+ 2. Lifecycle management extensions including suspend/resume and
22
+ restart.
23
+ 3. Improved error handling
24
+ 4. Indexed jobs
25
+ 5. Task dependencies
26
+ 2 . Scheduling extensions, e.g:
27
+ 1 . Co-scheduling
28
+ 2. Fair-share scheduling
29
+ 3. Queue scheduling
30
+ 4. Preemption and reclaims
31
+ 5. Reservartions and backfills
32
+ 6. Topology-based scheduling
33
+ 3 . Runtime extensions, e.g:
34
+ 1 . Support for specialized continer runtimes like Singularity,
35
+ with GPU accelerator extensions and enhanced security features.
36
+ 4 . Other
37
+ 1 . Data locality awareness and intelligent scheduling
38
+ 2. Optimizations for data throughput, round-trip latency, etc.
39
+
40
+
41
+ Volcano builds upon a decade and a half of experience running a wide
42
+ variety of high performance workloads at scale using several systems
43
+ and platforms, combined with best-of-breed ideas and practices from
44
+ the open source community.
10
45
11
46
## Overall Architecture
12
47
@@ -18,38 +53,51 @@ The easiest way to use Volcano is to use the Helm chart.
18
53
19
54
20
55
### 1. Volcano Image
21
- Official images now are not available on DockerHub, however you can build them locally with command:
56
+
57
+ Official images are not yet available on DockerHub, however you can
58
+ build them locally with the command:
59
+
22
60
```
23
61
make docker
24
62
```
63
+
25
64
** NOTE** : You need ensure the images are correctly loaded in your kubernetes cluster, for
26
65
example, if you are using [ kind cluster] ( https://github.com/kubernetes-sigs/kind ) ,
27
66
try command ``` kind load docker-image <image-name>:<tag> ``` for each of the images.
28
67
29
68
### 2. Helm charts
30
- First of all, clone repo to your local path
69
+ First of all, clone the repo to your local path:
70
+
31
71
```
32
72
# mkdir -p $GOPATH/src/volcano.sh/
33
73
# cd $GOPATH/src/volcano.sh/
34
74
# git clone https://github.com/volcano-sh/volcano.git
35
75
```
36
- Second, install required helm plugin and generate valid certificate, volcano uses a helm plugin ** gen-admission-secret**
37
- to generate certificate for admission service to communicate with kubernetes API server.
76
+
77
+ Second, install the required helm plugin and generate valid
78
+ certificate, volcano uses a helm plugin ** gen-admission-secret** to
79
+ generate certificate for admission service to communicate with
80
+ kubernetes API server.
81
+
38
82
```
39
83
#1. Install helm plugin
40
84
helm plugin install installer/chart/volcano/plugins/gen-admission-secret
41
85
42
86
#2. Generate secret within service name
43
87
helm gen-admission-secret --service <specified-name>-admission-service --namespace <namespace>
44
88
```
89
+
45
90
Finally, install helm chart.
91
+
46
92
```
47
93
helm install installer/chart/volcano --namespace <namespace> --name <specified-name>
48
94
```
95
+
49
96
** NOTE** : The ``` <specified-name> ``` used in the two commands above should be identical.
50
97
51
98
52
- To Verify your installation run the below commands
99
+ To Verify your installation run the following commands:
100
+
53
101
```
54
102
#1. Verify the Running Pods
55
103
# kubectl get pods --namespace <namespace>
0 commit comments