race condition in quadlet pod

[Xyz00777]

Hi, im not sure how to solve my problem.
I am using quadlet to manage a pod with 5 containers inside of it.
and one container have a dependency to start after a database.
I realized the dependency with systemd requires and systemd after, but now i have the problem, that systemd wants to start the webserver container inside the pod before the db container and it says there is no service with the name of the db container. if i restart the start service for the webserver a few moments later manual it knows the service and can start without any problem.

How can i solve this?

[ygalblum]

Not sure I understood all the details. Can you share your Quadlet files, or something to reproduce your issue.

Also, if the DB is a dependency, why not run it outside of the pod?

[Xyz00777]

this is the short version of my ansible script where just the pods and the container creation is in.
tldr:
i configured the pod to start after a specific mount is up (it looks like this works now), and than all container should start beside the paperless-webserver because it should start at last. now its the first time its not the db container but the tika container service who was not there.
here is the journal log:

systemctl --user status paperless-webserver.service
× paperless-webserver.service
     Loaded: loaded (/srv/paperless/.config/containers/systemd/paperless-webserver.container; generated)
     Active: failed (Result: exit-code) since Fri 2025-04-04 13:29:35 UTC; 1min 51s ago
    Process: 1635 ExecStart=/usr/bin/podman run --name=paperless-webserver --cidfile=/run/user/3001/paperless-webserver.cid --replace --rm --cgroups=split --sdnotify=conmon -d --user=1000:1000 --userns keep-id -v /opt/paperlessngx/data>
    Process: 1692 ExecStopPost=/usr/bin/podman rm -v -f -i --cidfile=/run/user/3001/paperless-webserver.cid (code=exited, status=0/SUCCESS)
   Main PID: 1635 (code=exited, status=125)
        CPU: 236ms

Apr 04 13:29:34 nextcloud1 systemd[1068]: Starting paperless-webserver.service...
Apr 04 13:29:34 nextcloud1 podman[1635]: 2025-04-04 13:29:34.817516022 +0000 UTC m=+0.110445768 image pull cf71d7b330d15de9eb08042419a400d20db29f2986d196daf3bd00d733372e76 ghcr.io/paperless-ngx/paperless-ngx:latest
Apr 04 13:29:34 nextcloud1 paperless-webserver[1635]: Error: "paperless-tika" is not a valid container, cannot be used as a dependency: no container with name or ID "paperless-tika" found: no such container
Apr 04 13:29:34 nextcloud1 systemd[1068]: paperless-webserver.service: Main process exited, code=exited, status=125/n/a
Apr 04 13:29:35 nextcloud1 systemd[1068]: paperless-webserver.service: Failed with result 'exit-code'.
Apr 04 13:29:35 nextcloud1 systemd[1068]: Failed to start paperless-webserver.service.

And here is my ansible config:

- name: Deploy paperless using Podman Quadlet in user mode
  hosts: nextcloud
  become: true
  become_user: paperless
  vars:
    target_user: paperless
  tasks:
    - name: Define the Paperless-ngx Pod
      containers.podman.podman_pod:
        name: paperless
        state: quadlet
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}.pod"
        generate_systemd:
          Requires: opt-paperlessngx.mount
          After: opt-paperlessngx.mount
        ports:
          - 8000:8000
        quadlet_options:
          - |
            [Install]
            WantedBy=default.target

    - name: Create Redis container
      containers.podman.podman_container:
        name: paperless-redis
        image: docker.io/library/redis:latest
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}-redis.container"
        pod: paperless.pod
        generate_systemd:
          restart_policy: unless-stopped
        state: quadlet
        quadlet_options:
          - AutoUpdate=registry
          - |
            [Install]
            WantedBy=multi-user.target

    - name: Create PostgreSQL container
      containers.podman.podman_container:
        name: paperless-postgresql
        image: docker.io/library/postgres:latest
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}-postgresql.container"
        pod: paperless.pod
        userns: keep-id
        user: 998:998
        generate_systemd:
          restart_policy: unless-stopped
        env:
          POSTGRES_USER: paperless
          POSTGRES_PASSWORD: paperless
          POSTGRES_DB: paperless
        volumes:
          - /opt/paperlessngx/pgdata:/var/lib/postgresql/data
        state: quadlet
        quadlet_options:
          - AutoUpdate=registry
          - |
            [Install]
            WantedBy=multi-user.target

    - name: Create Paperless-ngx webserver container
      containers.podman.podman_container:
        name: paperless-webserver
        image: ghcr.io/paperless-ngx/paperless-ngx:latest
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}-webserver.container"
        pod: paperless.pod
        userns: keep-id
        user: 1000:1000
        generate_systemd:
          restart_policy: unless-stopped
        env:
          PAPERLESS_REDIS: redis://paperless-redis:6379
          PAPERLESS_DBHOST: paperless-postgresql
          PAPERLESS_TIKA_ENABLED: 1
          PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://paperless-gotenberg:3000
          PAPERLESS_TIKA_ENDPOINT: http://paperless-tika:9998
          PAPERLESS_OCR_LANGUAGE: deu+eng
          PAPERLESS_URL: https://paperless.example.org
          PAPERLESS_SECRET_KEY: xxxx-xxxxx
          PAPERLESS_CONSUMER_RECURSIVE: "true"
          PAPERLESS_OCR_DESKEW: true
          PAPERLESS_OCR_CLEAN: clean-final
          PAPERLESS_TIME_ZONE: Europe/Berlin
          #     PAPERLESS_EMAIL_TASK_CRON: "*/1 * * * *" # default is 10min
          USERMAP_UID: "1000"
          USERMAP_GID: "1000"
        requires:
          - "{{ target_user }}-tika"
          - "{{ target_user }}-gotenberg"
          - "{{ target_user }}-redis"
          - "{{ target_user }}-postgresql"
        volumes:
          - /opt/paperlessngx/data:/usr/src/paperless/data
          - /opt/paperlessngx/media:/usr/src/paperless/media
          - /opt/paperlessngx/export:/usr/src/paperless/export
          - /opt/paperlessngx/consume:/usr/src/paperless/consume
        state: quadlet
        quadlet_options:
          - AutoUpdate=registry
          - |
            [Install]
            WantedBy=multi-user.target

    - name: Create Tika container
      containers.podman.podman_container:
        name: paperless-tika
        image: docker.io/apache/tika:latest
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}-tika.container"
        pod: paperless.pod
        generate_systemd:
          restart_policy: unless-stopped
        state: quadlet
        quadlet_options:
          - AutoUpdate=registry
          - |
            [Install]
            WantedBy=multi-user.target

    - name: Pull an image
      containers.podman.podman_image:
        name: docker.io/gotenberg/gotenberg:latest

    - name: Create Gotenberg container
      containers.podman.podman_container:
        name: paperless-gotenberg
        image: docker.io/gotenberg/gotenberg:latest
        image_strict: false
        quadlet_dir: ~/.config/containers/systemd/
        quadlet_file_mode: "0600"
        quadlet_filename: "{{ target_user }}-gotenberg.container"
        pod: paperless.pod
        generate_systemd:
          restart_policy: unless-stopped
        state: quadlet
        command: gotenberg --chromium-disable-javascript=true --chromium-allow-list=file:///tmp/.*

PS:
where is the difference between multi-user.target and default.target?
maybe you also find out why i cant download the gutenberg container with the containers.podman.podman_container module but with the containers.podman.podman_image :D im extremly confused and wasnt able to find out, because when the service want to download it than it everytime want to use the enforce short name even when like now the full download name is declared :D but that is not relevant atm

Thanks already for trying to help me :)