Possibility to persist data (keys, secrets, certs) automatically, not only with export/import

[JoeNeu]

Maybe i have overlooked things but it looks like the secrets stored in a vault are only in memory? It would be a nice feature if we could persist them in some sort of filebased db. Further this file could be exported (e.g. docker volume) and even shared for development reasons.

[nagyesta]

Hi @JoeNeu ,
thank you for reaching out, and thank you for using Lowkey Vault! You are correct, unfortunately this is not a feature yet. There was someone earlier asking for something similar, so I think there must be a valid use-case for this, but it comes with some trade-offs.

1. I think it would potentially encourage people to misuse the test double and really use it as a drop-in replacement for AKV which would be totally unsafe.
2. It would mean, that every change I do in-memory would need to happen in some sort of a transaction and the application state would need to be persisted to disk regularly. This is not trivial, but most certainly could be done, even if it would mean, that we would make things slower as the export process can be relatively slow due to the JSON conversion of a larger data-set.

With that said, I think this is still something I would consider in the long run so let me convert this discussion to an issue and put it to the backlog at least. That way you will see the updates if I will have some time to implement it...

Does it sound fair to you?

Thank you again and sorry for the bad news!

[nagyesta]

Hi @JoeNeu ,
this feature is being released in v3.1.0 today. Please feel free to try it and let me know whether it suitable for your use-case!
Thank you in advance!

[webartoli]

excellent news. I'm using lowkeyvault with docker and can't find what path I need to mount the volume on.

I have searched the documentation in the docker hub, on github and in the examples but I am not finding it.

Thanks to anyone who will help me

[nagyesta]

Hi @webartoli ,
thanks, you can find the Docker specific parameters here: https://github.com/nagyesta/lowkey-vault/tree/main/lowkey-vault-docker#exporting-vault-content-on-change
Please feel free to reach out in case it is not working as intended!

[webartoli]

The usecase I would like to cover is to have:

• a file (perhaps using a bind mount in read only) to define the initial state
• a volume to define the local persisted state.

The idea is to commit the initial development state to the repository and do the docker compose up.

Currently the initial file located in /import/secrets-config.json.hbs is as follows:

{
  "vaults": [
    {
      "attributes": {
        "baseUri": "https://feature1.vault.localtest.me:{{port}}",
        "recoveryLevel": "Recoverable+Purgeable",
        "recoverableDays": 90,
        "created": {{now 0}},
        "deleted": null
        }
    },
    {
      "attributes": {
        "baseUri": "https://feature2.vault.localtest.me:{{port}}",
        "recoveryLevel": "Recoverable+Purgeable",
        "recoverableDays": 90,
        "created": {{now 0}},
        "deleted": null
        },
      "secrets": {
        "secret-name": {
          "versions": [
            {
              "attributes": {
                "enabled": true,
                "created": {{now 0}},
                "updated": {{now 0}},
                "recoveryLevel": "Recoverable+Purgeable",
                "recoverableDays": 90
                },
              "vaultBaseUri": "https://feature2.vault.localtest.me:{{port}}",
              "entityId": "21d525d7-2ad3-4842-9ad0-000000000001",
              "entityVersion": "00000000000000000000000000000001",
              "managed": false,
              "value": "{\"Username\":\"admin\",\"Password\":\"supersecretpassword\"}",
              "contentType": "text/plain"
            }
          ]
        }
      }
    }
  ]
}

and docker compose is:

  lowkey-vault:
    image: nagyesta/lowkey-vault
    ports:
      - 8443:8443
    environment:
      LOWKEY_VAULT_NAMES: "-"
      LOWKEY_IMPORT_LOCATION: /import/secrets-config.json.hbs
      LOWKEY_IMPORT_TEMPLATE_PORT: 8443
    volumes:
      - ./infra/secrets-config.json.hbs:/import/secrets-config.json.hbs:r # initial import
    networks:
      default:
        aliases:
          - feature1.vault.localtest.me
          - feature2.vault.localtest.me

and I cannt find a way to achieve the goal

[nagyesta]

Hi @webartoli ,
thanks for the additional details! I have moved it to a new issue (#1545 ) to be able to continue without disturbing others on the thread.