Docker and scalability

[max06]

Description:

I'm currently trying to utilize scaled workers to distribute the workload, mainly for federation, to multiple processes. It turned out, this is more complicated than it needs to be.

See this example docker-compose file:

  synapse-worker-federation-sender:
    <<: *synapse
    command: >
      run 
      -m synapse.app.federation_sender
      --config-path=/data/homeserver.yaml
      --config-path=/data/federation_sender.yaml
    labels: [... stuff for reverse proxy ...]
    deploy:
      mode: replicated
      replicas: 4

As soon as I start running more than one instance, several issues arise:

• a worker_name needs to be specified for each worker
  • this needs to be done in the workers config file
  • the shared configuration needs to be updated as well
  • the main process needs to be restarted as well

Usually it's enough to have one side knowing the details for connecting to its peers. It's either the main process knowing about all worker instances, or all worker instance knowing who's the central controlling unit (preferred). Here we have both sides knowing about both ends, redundant information.

Suggestions:

• Make things like worker_name configurable via env vars - to avoid having different, but similar configuration files for the same purpose
• Allow using dynamically generated worker names if nothing is specified - the hostname would be a great start in case of docker containers
• Make workers register themselfes via redis pubsub at runtime.

I'll have to scale up my synapse configuration by a lot in the foreseeable future and I'd really prefer to not define every single worker instance manually. Having at least one of the above mentioned options would make it easier.

I'd also be very happy if I've simply overseen the obvious!

[anoadragon453]

Yes, it's known that this configuration dance is a pain point for large deployments, and it's something we'd like to resolve. Do take a look at some existing issues which touch on similar points: #11489, #12604.

I'm going to leave this issue open, as it provides a clear use case and suggestions for improvement.