Data lost, how is tandoor saving data?

[tueboesen]

I tried to migrate from postgres 16 to 17 the other day, which failed, and for some unknown reason all my backup snapshots also seems to be corrupted.

Now I just setup a dummy tandoor instance (I'm running everything through docker), and I saved a single new recipe into this. However, I do not see anything in the postgres database afterwards.

Here is what I see if I exec into the postgres docker container and query the database, after having setup my dummy cookbook with a single recipe called DRAGONS

As far as I can see everything looks fine in the logs, and I can access the recipe on my domain, but nothing is showing up in the postgres database. Furthermore, there is in fact not a single reference to DRAGONS anywhere in the persistent storage volume that is used for all the tandoor files, which I guess just means that it is stored in binary somehow...

Any insight or help would be greatly appreciated, since I want to ensure that everything is setup correctly this time and that I can restore from backup before I start using this again.

root@19498d93aeee:/# psql -U 1234 -d tandoor
psql (17.2 (Debian 17.2-1.pgdg120+1))
Type "help" for help.

tandoor=# \l
                                                 List of databases
   Name    | Owner | Encoding | Locale Provider |  Collate   |   Ctype    | Locale | ICU Rules | Access privileges 
-----------+-------+----------+-----------------+------------+------------+--------+-----------+-------------------
 postgres  | 1234  | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | 
 tandoor   | 1234  | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | 
 template0 | 1234  | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | =c/1234          +
           |       |          |                 |            |            |        |           | 1234=CTc/1234
 template1 | 1234  | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | =c/1234          +
           |       |          |                 |            |            |        |           | 1234=CTc/1234
(4 rows)

tandoor=# \dt
Did not find any relations.
tandoor=# \c tandoor
You are now connected to database "tandoor" as user "1234".
tandoor=# \dt
Did not find any relations.
tandoor=# \c postgres
You are now connected to database "postgres" as user "1234".
postgres=# \dt
Did not find any relations.
postgres=# 

My docker-compose.yml:

services:
  postgres_tandoor:
    container_name: postgres_tandoor
    restart: unless-stopped
    image: postgres:17.2@sha256:87ec5e0a167dc7d4831729f9e1d2ee7b8597dcc49ccd9e43cc5f89e808d2adae
    volumes:
      - ${LOCAL_BASE_PATH}/tandoor/postgresql:/var/lib/postgresql/data
    env_file:
      - .env
    environment:
        - POSTGRES_DB=${POSTGRES_DATABASE}
        - POSTGRES_USER=${POSTGRES_USER}
        - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      interval: 1s
      timeout: 5s
      retries: 10
    networks:
      - internal

  pgbackups:
      image: prodrigestivill/postgres-backup-local:17
      restart: always
      user: postgres:postgres # Optional: see below
      volumes:
          - ${LOCAL_BASE_PATH}/tandoor/pgbackups:/backups
      links:
          - postgres_tandoor
      depends_on:
          - postgres_tandoor
      env_file:
        - .env
      environment:
          - POSTGRES_HOST=postgres_tandoor
          - POSTGRES_DB=${POSTGRES_DATABASE}
          - POSTGRES_USER=${POSTGRES_USER}
          - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
       #  - POSTGRES_PASSWORD_FILE=/run/secrets/db_password <-- alternative for POSTGRES_PASSWORD (to use with docker secrets)
          - POSTGRES_EXTRA_OPTS=-Z1 --schema=public --blobs
          - SCHEDULE=@daily
          - BACKUP_ON_START=TRUE
          - BACKUP_KEEP_DAYS=7
          - BACKUP_KEEP_WEEKS=4
          - BACKUP_KEEP_MONTHS=6
          - HEALTHCHECK_PORT=8080
          - TZ=${TZ}
      networks:
        - internal


  web_recipes:
    restart: unless-stopped
    image: vabene1111/recipes:1.5.30@sha256:72d1e66294e4a28ba8e1587afd219f7106c0c786d9531ba15de5e14c0d62ad81
    env_file:
      - .env
    volumes:
      - ${LOCAL_BASE_PATH}/tandoor/staticfiles:/opt/recipes/staticfiles
      # Do not make this a bind mount, see https://docs.tandoor.dev/install/docker/#volumes-vs-bind-mounts
      - nginx_config:/opt/recipes/nginx/conf.d
      - ${LOCAL_BASE_PATH}/tandoor/mediafiles:/opt/recipes/mediafiles
    environment:
      - SECRET_KEY=#$tp%v6*(*ba01wcz(ip(i5vfz8z$f%qdio&q@anr1#$=%(m4d
    depends_on:
      - postgres_tandoor
    networks:
      - internal

  nginx_recipes:
    image: nginx:mainline-alpine@sha256:b471bb609adc83f73c2d95148cf1bd683408739a3c09c0afc666ea2af0037aef
    restart: unless-stopped
    ports:
      - ${PORT_TANDOOR}:80
    env_file:
      - .env
    depends_on:
      - web_recipes
    volumes:
      # Do not make this a bind mount, see https://docs.tandoor.dev/install/docker/#volumes-vs-bind-mounts
      - nginx_config:/etc/nginx/conf.d:ro
      - ${LOCAL_BASE_PATH}/tandoor/staticfiles:/static:ro
      - ${LOCAL_BASE_PATH}/tandoor/mediafiles:/media:ro
    networks:
      - proxy
      - internal
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy" # This was supposed to somehow make sure it got the external ip rather than the internal docker ip, but it does not seem to work
      - "traefik.http.routers.${TRAEFIK_SERVICE_NAME}.rule=Host(`tandoor.${DOMAIN}`) || Host(`food.${DOMAIN}`)"
      - "traefik.http.routers.${TRAEFIK_SERVICE_NAME}.entrypoints=https"
      - "traefik.http.routers.${TRAEFIK_SERVICE_NAME}.tls=true"
      - "traefik.http.services.${TRAEFIK_SERVICE_NAME}.loadbalancer.server.port=80"


volumes:
  nginx_config:
  staticfiles:

networks:
  proxy:
    external: true
  internal:
    external: false

This is what my docker log looks like

root@truenas[...DataStore/docker/docker_stack/tandoor]# docker compose up --remove-orphans
WARN[0000] The "tp" variable is not set. Defaulting to a blank string. 
WARN[0000] The "f" variable is not set. Defaulting to a blank string. 
[+] Running 4/0
 ✔ Container postgres_tandoor         Created                                                                                                                                                                                                                                                                            0.0s 
 ✔ Container tandoor-pgbackups-1      Created                                                                                                                                                                                                                                                                            0.0s 
 ✔ Container tandoor-web_recipes-1    Created                                                                                                                                                                                                                                                                            0.0s 
 ✔ Container tandoor-nginx_recipes-1  Created                                                                                                                                                                                                                                                                            0.0s 
Attaching to postgres_tandoor, nginx_recipes-1, pgbackups-1, web_recipes-1
postgres_tandoor  | 
postgres_tandoor  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_tandoor  | 
postgres_tandoor  | 2025-02-24 12:54:04.010 CET [1] LOG:  starting PostgreSQL 17.2 (Debian 17.2-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
postgres_tandoor  | 2025-02-24 12:54:04.010 CET [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgres_tandoor  | 2025-02-24 12:54:04.010 CET [1] LOG:  listening on IPv6 address "::", port 5432
postgres_tandoor  | 2025-02-24 12:54:04.045 CET [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_tandoor  | 2025-02-24 12:54:04.075 CET [30] LOG:  database system was shut down at 2025-02-24 12:54:00 CET
postgres_tandoor  | 2025-02-24 12:54:04.092 CET [1] LOG:  database system is ready to accept connections
web_recipes-1     | Checking configuration...
web_recipes-1     | Waiting for database to be ready...
web_recipes-1     | Database is ready
web_recipes-1     | Migrating database
pgbackups-1       | 2025/02/24 12:54:04 Running version: v0.0.11
pgbackups-1       | 2025/02/24 12:54:04 new cron: @daily
pgbackups-1       | 2025/02/24 12:54:04 Opening port 8080 for health checking
pgbackups-1       | 2025/02/24 12:54:04 18 cmd: /backup.sh 
pgbackups-1       | 2025/02/24 12:54:04 18: Creating dump of tandoor database from postgres_tandoor...
pgbackups-1       | 2025/02/24 12:54:05 18: Replacing daily backup /backups/daily/tandoor-20250224.sql.gz file this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/daily/tandoor-20250224.sql.gz' => '/backups/last/tandoor-20250224-125404.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Replacing weekly backup /backups/weekly/tandoor-202509.sql.gz file this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/weekly/tandoor-202509.sql.gz' => '/backups/last/tandoor-20250224-125404.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Replacing monthly backup /backups/monthly/tandoor-202502.sql.gz file this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/monthly/tandoor-202502.sql.gz' => '/backups/last/tandoor-20250224-125404.sql.gz'
pgbackups-1       | Point last backup file to this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/last/tandoor-latest.sql.gz' -> 'tandoor-20250224-125404.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Point latest daily backup to this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/daily/tandoor-latest.sql.gz' -> 'tandoor-20250224.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Point latest weekly backup to this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/weekly/tandoor-latest.sql.gz' -> 'tandoor-202509.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Point latest monthly backup to this last backup...
pgbackups-1       | 2025/02/24 12:54:05 18: '/backups/monthly/tandoor-latest.sql.gz' -> 'tandoor-202502.sql.gz'
pgbackups-1       | 2025/02/24 12:54:05 18: Cleaning older files for tandoor database from postgres_tandoor...
pgbackups-1       | 2025/02/24 12:54:05 18: SQL backup created successfully
nginx_recipes-1   | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_recipes-1   | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx_recipes-1   | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx_recipes-1   | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf is not a file or does not exist
nginx_recipes-1   | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
nginx_recipes-1   | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_recipes-1   | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx_recipes-1   | /docker-entrypoint.sh: Configuration complete; ready for start up
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: using the "epoll" event method
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: nginx/1.27.4
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: built by gcc 14.2.0 (Alpine 14.2.0) 
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: OS: Linux 6.6.44-production+truenas
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: start worker processes
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: start worker process 20
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: start worker process 21
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: start worker process 22
nginx_recipes-1   | 2025/02/24 12:54:05 [notice] 1#1: start worker process 23
web_recipes-1     | Operations to perform:
web_recipes-1     |   Apply all migrations: account, admin, auth, authtoken, contenttypes, cookbook, oauth2_provider, sessions, sites, socialaccount
web_recipes-1     | Running migrations:
web_recipes-1     |   No migrations to apply.
web_recipes-1     | Collecting static files, this may take a while...
web_recipes-1     | js-reverse file written to /opt/recipes/cookbook/static/django_js_reverse
web_recipes-1     | 
web_recipes-1     | 1 static file copied to '/opt/recipes/staticfiles', 575 unmodified, 1186 post-processed.
web_recipes-1     | Done
web_recipes-1     | [2025-02-24 12:54:14 +0100] [1] [INFO] Starting gunicorn 22.0.0
web_recipes-1     | [2025-02-24 12:54:14 +0100] [1] [INFO] Listening at: http://[::]:8080 (1)
web_recipes-1     | [2025-02-24 12:54:14 +0100] [1] [INFO] Using worker: gthread
web_recipes-1     | [2025-02-24 12:54:14 +0100] [23] [INFO] Booting worker with pid: 23
web_recipes-1     | [2025-02-24 12:54:14 +0100] [24] [INFO] Booting worker with pid: 24
web_recipes-1     | [2025-02-24 12:54:14 +0100] [25] [INFO] Booting worker with pid: 25

My .env file looks like this:

TRAEFIK_SERVICE_NAME=tandoor
POSTGRES_USER=1234
POSTGRES_PASSWORD=1234
POSTGRES_DATABASE=tandoor
PUID=568
PGID=568
TZ=Europe/Copenhagen
REPOSITORY_BASE_PATH=/mnt/DataStore/docker/docker_stack
LOCAL_BASE_PATH=/mnt/DataStore/docker/data

[smilerz]

do you have the postgres db, user and password defined in the .env? If it is defined you don't need to them on the postgres container (they are already in the .env)

also, your secret needs to be rotated - don't share them publicly.

[tueboesen]

I have all the variables defined in the env file, I just post them in like this to be more explicit.
But other than this it looks okay to you?
Because I'm not seeing any data in the postgres database when spinning things up using the above docker-compose.yml and I'm really wondering where the data is then ending up.

[smilerz]

can you please share the .env

[tueboesen]

My .env file looks like this:

TRAEFIK_SERVICE_NAME=tandoor
POSTGRES_USER=1234
POSTGRES_PASSWORD=1234
POSTGRES_DATABASE=tandoor
PUID=568
PGID=568
TZ=Europe/Copenhagen
REPOSITORY_BASE_PATH=/mnt/DataStore/docker/docker_stack
LOCAL_BASE_PATH=/mnt/DataStore/docker/data

[smilerz]

You're missing
DB_ENGINE=django.db.backends.postgresql

[tueboesen]

So wait, if I do not have that, then it doesn't post anything into the database? Where does the data then go?
I will try to add that immediately and report back

[smilerz]

By default, it will use a SQLite database

[tueboesen]

So it turns out I was indeed using SQLite as my backend database, which of course explains how I lost all my data, since it was accidentally saved in the SQLite database which was never mapped to a persistent volume.