feat: Configure micoservices queues using env variable #17675
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces fine-grained control over microservices queues using two new environment variables:
IMMICH_MICROSERVICES_QUEUES_INCLUDE
: Specify which queues to processIMMICH_MICROSERVICES_QUEUES_EXCLUDE
: Specify which queues to ignoreWhy
Many friends that I know run Immich on NAS devices like QNAP and Synology, which typically lack the computing power necessary for resource-intensive tasks such as video encoding and machine learning. This leads them to deploy the machine-learning container on a separate, more powerful machine.
In my personal setup, I run Immich on a QNAP NAS and initially tried moving all microservices to a more powerful homelab server with an AMD Zen9 processor. I mounted the QNAP share via Samba using autofs. While this approach significantly improved video transcoding performance, it introduced issues with external library watching and scanning due to inherent limitations of Samba shares.
The new queue-specific environment variables allow for perfect distribution:
NAS
services: immich-server: ... environment: IMMICH_WORKERS_INCLUDE: 'api' immich-microservices: ... environment: IMMICH_WORKERS_INCLUDE: 'microservices' + IMMICH_MICROSERVICES_QUEUES_EXCLUDE: 'videoConversion,thumbnailGeneration'
Server with GPU
services: immich-machine-learning: ... immich-microservices: ... environment: IMMICH_WORKERS_INCLUDE: 'microservices' + IMMICH_MICROSERVICES_QUEUES_INCLUDE: 'videoConversion,thumbnailGeneration'
Fixes # (issue)
How Has This Been Tested?
I implemented some unit test. Also I did perform manual test:
I tested this locally by running localy server twice:
Then I just opened web ui, found video run transcoding tasks for all. I should see video transcoding only in second process. All other tasks still worked as before.
Checklist:
src/services/
uses repositories implementations for database calls, filesystem operations, etc.src/repositories/
is pretty basic/simple and does not have any immich specific logic (that belongs insrc/services/
)