Skip to content

Commit b25221a

Browse files
intrandAndrew Suderman
and
Andrew Suderman
authored
support argocd application generation via course file (#596)
* generate and write argocd application manifests * whoops :) * avoid accidental double-loop * reuse writeYAML() * commented imports from argocd directly * add gitops to releases * merge+override release gitops over global gitops * remove obsolete comment * use yaml paths in warnings * add some basic documentation for gitops * go mod tidy * clarify commented imports w/ future instructions * move gitops/argocd stuff to TemplateRelease() * lowercase argo app names too * put warnings behind verbosity 3 * skip when app.metadata.name is missing * create both outputdir and appsoutputdir if missing * skip argocd app gen when --output-dir is not given * remove obsolete comment * clarify --output-dir and gitops interaction in doc * remove another obsolete comment * pass --output-dir value through reckoner.Client Co-authored-by: Andrew Suderman <[email protected]>
1 parent f552afb commit b25221a

File tree

9 files changed

+279
-17
lines changed

9 files changed

+279
-17
lines changed

cmd/root.go

+2-1
Original file line numberDiff line numberDiff line change
@@ -162,14 +162,15 @@ var templateCmd = &cobra.Command{
162162
CreateNamespaces: false,
163163
ContinueOnError: continueOnError,
164164
Releases: onlyRun,
165+
OutputDirectory: templateOutputDir,
165166
}
166167

167168
err := client.Init(courseFile, false)
168169
if err != nil {
169170
color.Red(err.Error())
170171
os.Exit(1)
171172
}
172-
tmpl, err := client.TemplateAll(templateOutputDir)
173+
tmpl, err := client.TemplateAll()
173174
if err != nil {
174175
color.Red(err.Error())
175176
os.Exit(1)

docs/usage.md

+52
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,10 @@ We'll be breaking this documentation down into sections to make reference easier
3636
- `secrets` _(list of objects)_
3737
A list of objects that define where and how to get secrets from your secret backend
3838
Required keys are `name` and `backend`.
39+
- `gitops` _(object)_
40+
A key of `argocd` is supported, which should contain an ArgoCD Application custom resource. When used at this top level,
41+
all releases will have an Application custom resource generated with inherited values. If you wish to override something
42+
in a particular release, repeat this same structure within the release item.
3943

4044
Example:
4145
```yaml
@@ -95,6 +99,8 @@ The `charts` block in your course define all the charts you'd like to install an
9599
Translates into a direct YAML values file for use with `-f <tmpfile>` for the helm install command line arguments
96100
- `plugins` _(string)_
97101
Prepend your helm commands with this `plugins` string (see [PR 99](https://github.com/FairwindsOps/reckoner/pull/99))
102+
- `gitops` _(object)_
103+
Same as top level `gitops` field, but overrides the top-level, if it exists, on a per-release basis.
98104

99105
```yaml
100106
...
@@ -263,6 +269,52 @@ secrets:
263269
- "changemeagain"
264270
```
265271

272+
## Gitops
273+
274+
Specifying the appropriate values within the `gitops` field will cause `reckoner` to generate the corresponding custom resources for popular GitOps agents. As of the writing, only ArgoCD is supported, although we may include support for other GitOps agents in the future.
275+
276+
Assuming you have the following YAML in the top level of your course file, you should see some basic ArgoCD Application custom resources being generated:
277+
```yaml
278+
gitops:
279+
argocd:
280+
spec:
281+
source:
282+
repoURL: https://gitlab.company.tld/organization/repository.git
283+
```
284+
285+
> Must be used with `reckoner template --output-dir <some_dir>` to produce any files.
286+
287+
For each release, the gitops.argocd.spec.destination namespace value will be read from the course file top-level namespace field, or from the release namespace field, if defined. See Namespace Management section for further details.
288+
289+
Another example which uses far more features of the ArgoCD agent is shown below. If you find a particular feature is missing, please open an issue with the project so we may discuss it.
290+
291+
Example:
292+
```yaml
293+
gitops:
294+
argocd:
295+
kind: Application
296+
apiVersion: argoproj.io/v1alpha1
297+
metadata:
298+
namespace: argocd
299+
annotations: {}
300+
spec:
301+
destination:
302+
server: https://kubernetes.default.svc
303+
project: default
304+
source:
305+
repoURL: https://github.com/someuser/clustername.git
306+
directory:
307+
recurse: true
308+
syncPolicy:
309+
automated:
310+
prune: true
311+
syncOptions:
312+
- CreateNamespace=true
313+
- PruneLast=true
314+
```
315+
316+
> Must be used with `reckoner template --output-dir <some_dir>` to produce any files.
317+
266318
## CLI Usage
267319

268320
```text

go.mod

+1-1
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ require (
99
github.com/fatih/color v1.13.0
1010
github.com/go-git/go-git/v5 v5.4.2
1111
github.com/gookit/color v1.5.1
12+
github.com/imdario/mergo v0.3.12
1213
github.com/mattn/go-colorable v0.1.12
1314
github.com/rhysd/go-github-selfupdate v1.2.3
1415
github.com/sergi/go-diff v1.1.0
@@ -54,7 +55,6 @@ require (
5455
github.com/google/go-github/v30 v30.1.0 // indirect
5556
github.com/google/go-querystring v1.1.0 // indirect
5657
github.com/google/gofuzz v1.1.0 // indirect
57-
github.com/imdario/mergo v0.3.12 // indirect
5858
github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf // indirect
5959
github.com/inconshreveable/mousetrap v1.0.0 // indirect
6060
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect

pkg/course/argocd.go

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
package course
2+
3+
// We may use these imports to track ArgoCD Applications exactly. However,
4+
// this also pulls in kubernetes api packages, which makes reckoner rely
5+
// on particular versions of kubernetes. At the time of writing, no such
6+
// relationship exists. Once we've firmly chosen a path, this comment
7+
// should be removed, and potentially this entire file.
8+
// import (
9+
// argoAppv1alpha1 "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
10+
// metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
11+
// )
12+
13+
type ArgoApplicationSpecSyncPolicyAutomated struct {
14+
Prune bool `yaml:"prune,omitempty"`
15+
}
16+
17+
type ArgoApplicationSpecSyncPolicy struct {
18+
Automated ArgoApplicationSpecSyncPolicyAutomated `yaml:"automated,omitempty"`
19+
Options []string `yaml:"syncOptions,omitempty"`
20+
}
21+
22+
type ArgoApplicationSpecSourceDirectory struct {
23+
Recurse bool `yaml:"recurse,omitempty"`
24+
}
25+
26+
type ArgoApplicationSpecSource struct {
27+
Directory ArgoApplicationSpecSourceDirectory `yaml:"directory,omitempty"`
28+
Path string `yaml:"path"`
29+
RepoURL string `yaml:"repoURL"`
30+
}
31+
32+
type ArgoApplicationSpecDestination struct {
33+
Server string `yaml:"server,omitempty"`
34+
Namespace string `yaml:"namespace,omitempty"`
35+
}
36+
37+
type ArgoApplicationSpec struct {
38+
Source ArgoApplicationSpecSource `yaml:"source"`
39+
Destination ArgoApplicationSpecDestination `yaml:"destination"`
40+
Project string `yaml:"project"`
41+
SyncPolicy ArgoApplicationSpecSyncPolicy `yaml:"syncPolicy,omitempty"`
42+
}
43+
44+
// ArgoApplicationMetadata contains the k8s metadata for the gitops agent CustomResource.
45+
// This is the resource/manifest/config the agent will read in, not the resources deployed by the agent.
46+
type ArgoApplicationMetadata struct {
47+
Name string `yaml:"name"`
48+
Namespace string `yaml:"namespace,omitempty"`
49+
Annotations map[string]string `yaml:"annotations,omitempty"`
50+
Labels map[string]string `yaml:"labels,omitempty"`
51+
}
52+
53+
type ArgoApplication struct {
54+
Kind string `yaml:"kind"`
55+
APIVersion string `yaml:"apiVersion"`
56+
Metadata ArgoApplicationMetadata `yaml:"metadata"`
57+
Spec ArgoApplicationSpec `yaml:"spec"`
58+
}

pkg/course/course.go

+10
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,15 @@ type FileV2 struct {
6868
Releases []*Release `yaml:"releases,omitempty" json:"releases,omitempty"`
6969
// HelmArgs is a list of arguments to pass to helm commands
7070
HelmArgs []string `yaml:"helm_args,omitempty" json:"helm_args,omitempty"`
71+
GitOps GitOps `yaml:"gitops,omitempty" json:"gitops,omitempty"`
72+
}
73+
74+
// GitOps is a field on the root of the course.yaml file which instructs reckoner to
75+
// generate CustomResources appropriate to the configured flavor of gitops agent.
76+
// For instance, if gitops.argocd is present and complete, ArgoCD Application resources
77+
// will be generated for each release in the course file with the corresponding values.
78+
type GitOps struct {
79+
ArgoCD ArgoApplication `yaml:"argocd" json:"argocd"`
7180
}
7281

7382
type NamespaceMgmt struct {
@@ -145,6 +154,7 @@ type Release struct {
145154
// Values contains any values that you wish to pass to the release. Everything
146155
// underneath this key will placed in a temporary yaml file and passed to helm as a values file.
147156
Values map[string]interface{} `yaml:"values,omitempty" json:"values,omitempty"`
157+
GitOps GitOps `yaml:"gitops,omitempty" json:"gitops,omitempty"`
148158
}
149159

150160
// ReleaseV1 represents a helm release and all of its configuration from v1 schema

pkg/course/coursev2.schema.json

+11-6
Original file line numberDiff line numberDiff line change
@@ -37,10 +37,13 @@
3737
"type": "array"
3838
},
3939
"pre_install": {
40-
"type": "array"
40+
"type": "array"
4141
}
4242
}
4343
},
44+
"gitops": {
45+
"type": "object"
46+
},
4447
"release": {
4548
"type": "array",
4649
"additionalProperties": {
@@ -159,6 +162,9 @@
159162
"hooks": {
160163
"$ref": "#/definitions/hooks"
161164
},
165+
"gitops": {
166+
"$ref": "#/definitions/gitops"
167+
},
162168
"minimum_versions": {
163169
"type": "object",
164170
"additionalProperties": false,
@@ -195,23 +201,22 @@
195201
"type": "object",
196202
"additionalProperties": true,
197203
"properties": {
198-
"backend":{
204+
"backend": {
199205
"type": "string"
200206
},
201207
"name": {
202208
"type": "string"
203209
}
204210
},
205211
"required": [
206-
"backend",
207-
"name"
212+
"backend",
213+
"name"
208214
]
209215
}
210-
211216
}
212217
},
213218
"required": [
214219
"namespace",
215220
"releases"
216221
]
217-
}
222+
}

pkg/reckoner/argocd.go

+120
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
package reckoner
2+
3+
import (
4+
"bytes"
5+
"errors"
6+
"os"
7+
"strings"
8+
9+
"github.com/fairwindsops/reckoner/pkg/course"
10+
"github.com/fatih/color"
11+
"github.com/imdario/mergo"
12+
"gopkg.in/yaml.v3"
13+
"k8s.io/klog/v2"
14+
)
15+
16+
func generateArgoApplication(release course.Release, courseFile course.FileV2) (app course.ArgoApplication, err error) {
17+
app = courseFile.GitOps.ArgoCD // use global config at root of course file
18+
19+
// if release.GitOps.ArgoCD.<whatever> exists, override the app.<whatever> with that one, recursively.
20+
err = mergo.Merge(&app, release.GitOps.ArgoCD, mergo.WithOverride)
21+
if err != nil {
22+
return app, err
23+
}
24+
25+
// default to a kind of Application if it was omitted in the course file
26+
if app.Kind == "" {
27+
app.Kind = "Application"
28+
}
29+
30+
// default to an API version of v1alpha1 if it was omitted in the course file
31+
if app.APIVersion == "" {
32+
app.APIVersion = "argoproj.io/v1alpha1"
33+
}
34+
35+
// unless they overrode it in the course file, assume the name of the argocd app is the same as the helm release
36+
// app:release should be a 1:1
37+
if app.Metadata.Name == "" {
38+
app.Metadata.Name = release.Name
39+
}
40+
41+
// Application.Metadata.Namespace is where the ArgoCD Application resource will go (not the helm release)
42+
if app.Metadata.Namespace == "" {
43+
klog.V(3).Infoln("No namespace declared in course file. Your ArgoCD Application manifests will likely get applied to the agent's default context.")
44+
}
45+
46+
// default source path to release name
47+
if app.Spec.Source.Path == "" {
48+
klog.V(3).Infoln("No .gitops.argocd.spec.source.path declared in course file for " + release.Name + ". The path has been set to its name.")
49+
app.Spec.Source.Path = release.Name
50+
}
51+
52+
// don't support ArgoCD Application spec.destination.namespace at all
53+
if app.Spec.Destination.Namespace != "" {
54+
klog.V(3).Infoln("Refusing to respect the .gitops.argocd.spec.destination.namespace value declared in course file for " + release.Name + ". Using the release namespace instead, if it exists. If none is specified, the default at the root of the course YAML file will be used. Remove the namespace from the ArgoCD destination to stop seeing this warning.")
55+
}
56+
57+
// Application.Spec.Destination.Namespace is where the helm releases will be applied
58+
if release.Namespace != "" { // there's a specific namespace for this release
59+
app.Spec.Destination.Namespace = release.Namespace // specify it as the destination namespace
60+
} else { // nothing was specified in the release
61+
app.Spec.Destination.Namespace = courseFile.DefaultNamespace // use the default namespace at the root of the course file
62+
}
63+
64+
if app.Spec.Destination.Server == "" {
65+
klog.V(3).Infoln("No .gitops.argocd.spec.destination.server declared in course file for " + release.Name + ". Assuming you want the kubernetes service in the default namespace. Specify to make this warning go away.")
66+
app.Spec.Destination.Server = "https://kubernetes.default.svc"
67+
}
68+
69+
if app.Spec.Project == "" {
70+
klog.V(3).Infoln("No .gitops.argocd.spec.project declared in course file for " + release.Name + ". We'll set it to a sensible default value of 'default'. Specify to make this warning go away.")
71+
app.Spec.Project = "default"
72+
}
73+
74+
return app, err
75+
}
76+
77+
func (c *Client) WriteArgoApplications(outputDir string) (err error) {
78+
appsOutputDir := outputDir + "/argocd-apps"
79+
for _, dir := range []string{outputDir, appsOutputDir} {
80+
if _, err := os.Stat(dir); errors.Is(err, os.ErrNotExist) {
81+
err := os.Mkdir(dir, os.ModePerm)
82+
if err != nil {
83+
return err
84+
}
85+
}
86+
}
87+
88+
for _, release := range c.CourseFile.Releases {
89+
// generate an argocd application resource
90+
app, err := generateArgoApplication(*release, c.CourseFile)
91+
if err != nil {
92+
return err
93+
}
94+
95+
if app.Metadata.Name == "" {
96+
color.Yellow("No metadata found for release " + release.Name + ". Skipping ArgoCD Applicationgeneration...")
97+
continue
98+
}
99+
100+
// generate name of app file
101+
appOutputFile := appsOutputDir + "/" + strings.ToLower(app.Metadata.Name) + ".yaml"
102+
103+
// prepare to write stuff (pretty)
104+
var b bytes.Buffer // used for encoding & return
105+
yamlEncoder := yaml.NewEncoder(&b) // create an encoder to handle custom configuration
106+
yamlEncoder.SetIndent(2) // people expect two-space indents instead of the default four
107+
err = yamlEncoder.Encode(&app) // encode proper YAML into slice of bytes
108+
if err != nil { // check for errors
109+
return err // bubble up
110+
}
111+
112+
// write stuff
113+
err = writeYAML(b.Bytes(), appOutputFile)
114+
if err != nil { // check for errors
115+
return err // bubble up
116+
}
117+
}
118+
119+
return err
120+
}

pkg/reckoner/client.go

+2
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ type Client struct {
6363
HelmArgs []string
6464
// Schema is a byte slice representation of the coursev2 json schema
6565
Schema []byte
66+
// OutputDirectory is the directory which will contain each YAML manifest in a separate file
67+
OutputDirectory string
6668
}
6769

6870
var once sync.Once

0 commit comments

Comments
 (0)