bring-your-own-db.md

Connecting to a managed SQL instance

We ship airbyte with the zalando postgres operator's db for persistence by default. This provides a lot of the benefits of a managed postgres instance at a lower cost, but if you'd rather use a familiar service like RDS this is still possible. You'll need to do a few things:

edit context.yaml

At the root of the repo, edit the context.yaml field and set configuration.airbyte.postgresDisabled: true, this will allow us to reconfigure airbyte for bring-your-own-db.

save the database password to a secret

you can use a number of methods for this, but simply adding a secret file as airbyte/helm/airbyte/templates/db-password.yaml like:

apiVersion: v1
kind: Secret
metadata:
  name: airbyte-db-password
stringData:
  password: {{ .Values.externalDb.password }}

Note: this password needs to be in the airbyte namespace. If you put it in our wrapper helm chart, that will be done by default for you.

modify airbyte's helm values.yaml

If you go to airbyte/helm/airbyte/values.yaml you'll need to provide credentials for postgres. They should look something like:

externalDb:
  password: <my password>
global:
  database:
    secretName: airbyte-db-password
    secretValue: password
airbyte:
  airbyte:
    externalDatabase:
      database: <YOUR_DB_NAME>
      host: <YOUR_DB_URL>
      user: <YOU_DB_USER>
      port: 5432

(we're ultimately beholden to the structure defined in airbyte's upstream helm chart here)

redeploy

From there, you should be able to run plural build --only airbyte && plural deploy --commit "using existing postgres instance" to use the managed sql instance