Skip to content

Commit a8245e8

Browse files
author
dexter
committed
Merge branch 'release/0.1.0'
2 parents a1059a0 + e683372 commit a8245e8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+6393
-0
lines changed

‎.dockerignore

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
/build
2+
/Dockerfile
3+
/.pipeline.yml
4+
/.gitignore
5+
/kubernetes.yml
6+
/README.md
7+
/dockerfiles

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/build

‎.pipeline.yml

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
apiVersion: v1alpha1
3+
kind: Pipeline
4+
metadata:
5+
name: kontinuous
6+
namespace: acaleph
7+
spec:
8+
selector:
9+
matchLabels:
10+
app: kontinuous
11+
type: ci-cd
12+
template:
13+
metadata:
14+
name: kontinuous
15+
labels:
16+
app: kontinuous
17+
type: ci-cd
18+
stages:
19+
- name: Build Docker Image
20+
type: docker_build
21+
- name: Unit Test
22+
type: command
23+
params:
24+
command:
25+
- make
26+
- test
27+
- name: Publish to Quay
28+
type: docker_publish
29+
params:
30+
external_registry: quay.io
31+
external_image_name: acaleph/kontinuous
32+
require_credentials: "TRUE"
33+
username: user
34+
password: password
35+
email: email
36+
secrets:
37+
- docker-credentials

‎Dockerfile

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
FROM golang:1.6
2+
3+
ENV GOPATH /go
4+
ENV SWAGGER_UI /swagger/dist
5+
6+
ADD . /go/src/github.com/AcalephStorage/kontinuous
7+
WORKDIR /go/src/github.com/AcalephStorage/kontinuous
8+
9+
RUN mkdir /swagger && tar xvzf third_party/swagger.tar.gz -C /swagger
10+
11+
RUN make
12+
13+
ENTRYPOINT build/bin/kontinuous

‎Makefile

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
APP_NAME = kontinuous
2+
3+
all: clean deps build
4+
5+
clean:
6+
@echo "--> Cleaning..."
7+
@rm -rfv ./build
8+
9+
format:
10+
@echo "--> Formatting..."
11+
@go fmt ./...
12+
13+
deps:
14+
@echo "--> Getting dependencies..."
15+
@go get -v -d ./...
16+
@go get -v github.com/golang/lint/golint
17+
18+
test: format
19+
@echo "--> Testing..."
20+
@go test -v ./...
21+
22+
lint:
23+
@echo "--> Running go lint..."
24+
golint ./...
25+
26+
27+
build: format
28+
@echo "--> Building..."
29+
@mkdir -p build/bin
30+
@go build -v -o build/bin/${APP_NAME} ./cmd
31+
@go build -v -o build/bin/${APP_NAME}-cli ./cli
32+
33+
package: build
34+
@echo "--> Packaging..."
35+
@mkdir -p build/tar
36+
@tar czf ./build/tar/${APP_NAME}-`go env GOOS`-`go env GOARCH`.tar.gz ./build/bin/${APP_NAME}
37+
@tar czf ./build/tar/${APP_NAME}-cli-`go env GOOS`-`go env GOARCH`.tar.gz ./build/bin/${APP_NAME}-cli

‎README.md

+239
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
KONTINUOUS
2+
==========
3+
4+
Kontinuous is a Continuous Integration & Delivery pipeline tool built specifically for Kubernetes. It aims to provide a platform for building applications using native Kubernetes Jobs and Pods.
5+
6+
## Running Kontinuous
7+
8+
### Dependencies
9+
10+
Running kontinuous requires the following to be setup:
11+
12+
- **etcd**
13+
14+
`etcd` is used as a backend for storing pipeline and build details. This is a dedicated instance to avoid issues with the Kubernetes etcd cluster.
15+
16+
- **minio**
17+
18+
`minio` is used to store the logs and artifacts. S3 could also be used as it is compatible with `minio`, although this has not been tested yet.
19+
20+
- **docker registry**
21+
22+
`registry` is used to store internal docker images.
23+
24+
### Running in Kubernetes
25+
26+
Kontinuous is meant to run inside a kubernetes cluster, preferrably by a Replication Controller.
27+
28+
The docker image can be found here: [quay.io/acaleph/kontinuous](quay.io/acaleph/kontinuous)
29+
30+
The following environment variables needs to be defined:
31+
32+
| Environment Variable | Description | Example |
33+
|----------------------|-----------------------------------------|------------------------|
34+
| KV_ADDRESS | The etcd address | etcd:2379 |
35+
| S3_URL | The minio address | http://minio:9000 |
36+
| KONTINUOUS_URL | The address where kontinuous is running | http://kontinuous:3005 |
37+
| INTERNAL_REGISTRY | The internal registry address | internal-registry:5000 |
38+
39+
A Kubernetes Secret also needs to be defined and mounted to the Pod. The secret should have a key named `kontinuous-secrets` and should contain the following data (must be base64 encoded):
40+
41+
```
42+
{
43+
"AuthSecret": "base64 encoded auth secret",
44+
"S3SecretKey": "s3 secret key",
45+
"S3AccessKey": "s3 access key"
46+
}
47+
```
48+
49+
`AuthSecret` is the secret for authenticating requests. This is needed by the clients to communicate with kontinuous through JWT.
50+
51+
`S3SecretKey` and `S3AccessKey` are the keys needed to access minio (or S3).
52+
53+
The secret needs to be mounted to the Pod to the path `/.secret`.
54+
55+
A sample yaml file for running kontinuous can be found [here](./k8s-spec.yml.example).
56+
57+
## Using Kontinuous
58+
59+
### Preparing the repository
60+
61+
#### Pipeline Spec
62+
63+
The repository needs to define a build pipeline in the repository root called `.pipeline.yml`
64+
65+
Here's a sample `.pipeline.yml`:
66+
67+
```
68+
---
69+
apiVersion: v1alpha1
70+
kind: Pipeline
71+
metadata:
72+
name: kontinuous
73+
namespace: acaleph
74+
spec:
75+
selector:
76+
matchLabels:
77+
app: kontinuous
78+
type: ci-cd
79+
template:
80+
metadata:
81+
name: kontinuous
82+
labels:
83+
app: kontinuous
84+
type: ci-cd
85+
stages:
86+
- name: Build Docker Image
87+
type: docker_build
88+
- name: Unit Test
89+
type: command
90+
params:
91+
command:
92+
- make
93+
- test
94+
- name: Publish to Quay
95+
type: docker_publish
96+
params:
97+
external_registry: quay.io
98+
external_image_name: acaleph/kontinuous
99+
require_credentials: "TRUE"
100+
username: user # taken from secret
101+
password: password # taken from secret
102+
email: email # taken from secret
103+
secrets:
104+
- docker-credentials
105+
```
106+
107+
The format is something similar to K8s Specs. Here are more details on some of the fields:
108+
109+
- `namespace` - the namespace to run the build
110+
- `matchLabels`/`labels` - the labels that are used for building the job
111+
- `stages` - defines the stages of the build pipeline
112+
113+
The general definition of a stage is:
114+
115+
```
116+
name: Friendly name
117+
type: {docker_build,command,docker_publish}
118+
params:
119+
key: value
120+
secrets:
121+
- secret-name
122+
```
123+
124+
- `type` can be: `docker_build`, `docker_publish`, or `command`.
125+
- `params` is a map of parameters to be loaded as environment variables.
126+
- `secrets` is a list of secrets that will be used as values for `params`.
127+
128+
#### Stages
129+
130+
`docker_build` can work without additional params. By default, it uses the `Dockerfile` inside the repository root. Optional params are:
131+
132+
- `dockerfile_path` - the path where the Dockerfile is located
133+
- `dockerfile_name` - the file name of the Dockerfile
134+
135+
After a build, the image is stored inside the internal docker registry.
136+
137+
`docker_publish` requires the following params:
138+
139+
- `external_registry` - the external registry name (eg. quay.io)
140+
- `external_image_name` - the name of the image (eg. acaleph/kontinuous)
141+
142+
Optional params:
143+
144+
- `require_crendentials` - defaults to `false`. Set to `true` if registry requires authentication
145+
- `username` - the username. this should be a key from one of the secrets file defined
146+
- `password` - the password. this should be a key from one of the secrets file defined
147+
- `email` - the email. this should be a key from one of the secrets file
148+
149+
The image that will be pushed is the image that was previously built. This does not work for now if no image was created.
150+
151+
`command` runs a command on the newly create docker image or on the image specified. Required param is `command` which is a list of string defining the command to execute.
152+
153+
Optional params are:
154+
155+
- `args` - a list of string to serve as the arguments for the command
156+
- `image` - the image to run the commands in. If not specified, the previous built image will be used.
157+
158+
159+
### Authentication
160+
161+
#### Github Token
162+
163+
Currently, only Github Repositories are supported. A github token needs to be generated in order to access the repositories.
164+
165+
To generate a github token, follow this [link](https://github.com/settings/tokens/new).
166+
167+
Make sure to enable access to the following:
168+
169+
- repo
170+
- admin:repo_hook
171+
- user
172+
173+
174+
#### JSON Web Token
175+
176+
Kontinuous uses JWT for authentication. To create a token, the `AuthSecret` (from kontinuous-secret) and the github token is required. One way of generating the token is using [jwt.io](https://jwt.io).
177+
178+
The header should be:
179+
180+
```
181+
{
182+
"alg": "HS256",
183+
"typ": "JWT"
184+
}
185+
```
186+
187+
Payload:
188+
189+
```
190+
{
191+
"identities": [
192+
{
193+
"access_token": "github token"
194+
}
195+
]
196+
}
197+
```
198+
199+
and Signature:
200+
201+
```
202+
HMACSHA256(
203+
base64UrlEncode(header) + "." +
204+
base64UrlEncode(payload),
205+
AuthSecret
206+
)
207+
208+
[x]secret base64 encoded
209+
```
210+
211+
Once a token is generated, this can be added to the request header as `Authorization: Bearer {token}` to authenticate requests.
212+
213+
## API
214+
215+
kontinuous is accessible from it's API. The API can be available via swagger.
216+
217+
The API doc can be accessed via `{kontinuous-address}/apidocs`
218+
219+
## Clients
220+
221+
At the moment, there is only a cli client (here)[https://github.com/AcalephStorage/kontinuous/tree/develop/cli].
222+
223+
## Development
224+
225+
Building `kontinuous` from source is done by:
226+
227+
```
228+
$ make deps build
229+
```
230+
231+
Build the docker image:
232+
233+
```
234+
$ docker build -t {tag} .
235+
```
236+
237+
## Notes
238+
239+
This is a Work In Progress designed to gather feedback from the community and has very basic functionality. Please file Issues (or better yet PRs!) so we can build the :ok_hand: CI/CD platform for K8s

0 commit comments

Comments
 (0)