Skip to content
This repository was archived by the owner on Apr 26, 2024. It is now read-only.

Support registering Application Services when running with workers under Complement. #12826

Merged
merged 5 commits into from
May 23, 2022
Merged

Support registering Application Services when running with workers under Complement. #12826

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
442d0f5
Register Complement's application services when running with workers
reivilibre May 20, 2022
66a3f2f
Newsfile
reivilibre May 20, 2022
7c4c366
Update docker/conf-workers/shared.yaml.j2
reivilibre May 23, 2022
59bfdb9
absolute -> resolve
reivilibre May 23, 2022
d461762
Merge branch 'develop' into rei/wcom_importhistory_as_reg
reivilibre May 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/12826.misc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Support registering Application Services when running with workers under Complement.
6 changes: 6 additions & 0 deletions docker/complement/conf-workers/start-complement-synapse-workers.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ export SYNAPSE_WORKER_TYPES="\
appservice, \
pusher"

# Add Complement's appservice registration directory, if there is one
# (It can be absent when there are no application services in this test!)
if [ -d /complement/appservice ]; then
export SYNAPSE_AS_REGISTRATION_DIR=/complement/appservice
fi

# Run the script that writes the necessary config files and starts supervisord, which in turn
# starts everything else
exec /configure_workers_and_start.py
11 changes: 10 additions & 1 deletion docker/conf-workers/shared.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,13 @@
redis:
enabled: true

{{ shared_worker_config }}
## Application Services ##
{% if appservice_registrations is not none %}
# A list of application service config files to use.
app_service_config_files:
{%- for path in appservice_registrations %}
- "{{ path }}"
{%- endfor %}
{%- endif %}

{{ shared_worker_config }}
15 changes: 15 additions & 0 deletions docker/configure_workers_and_start.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
# * SYNAPSE_REPORT_STATS: Whether to report stats.
# * SYNAPSE_WORKER_TYPES: A comma separated list of worker names as specified in WORKER_CONFIG
# below. Leave empty for no workers, or set to '*' for all possible workers.
# * SYNAPSE_AS_REGISTRATION_DIR: If specified, a directory in which .yaml and .yml files
# will be treated as Application Service registration files.
#
# NOTE: According to Complement's ENTRYPOINT expectations for a homeserver image (as defined
# in the project's README), this script may be run multiple times, and functionality should
Expand All @@ -29,6 +31,7 @@
import os
import subprocess
import sys
from pathlib import Path
from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Set

import jinja2
Expand Down Expand Up @@ -488,11 +491,23 @@ def generate_worker_files(
master_log_config = generate_worker_log_config(environ, "master", data_dir)
shared_config["log_config"] = master_log_config

# Find application service registrations
appservice_registrations = None
appservice_registration_dir = os.environ.get("SYNAPSE_AS_REGISTRATION_DIR")
if appservice_registration_dir:
# Scan for all YAML files that should be application service registrations.
appservice_registrations = [
str(reg_path.absolute())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find any documentation for absolute(). Kind of sounds like resolve()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, weird

>>> from pathlib import Path
>>> p = Path("/tmp")
>>> p.absolute()
PosixPath('/tmp')
>>> p.__class__
<class 'pathlib.PosixPath'>
>>> help(p.absolute)
Help on method absolute in module pathlib:

absolute() method of pathlib.PosixPath instance
    Return an absolute version of this path.  This function works
    even if the path doesn't point to anything.
    
    No normalization is done, i.e. all '.' and '..' will be kept along.
    Use resolve() to get the canonical path to a file.

Nothing against using resolve, this just seemed more intuitive to use (no idea why it's not documented!)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is likely fine, was just confused. 🤷

for reg_path in Path(appservice_registration_dir).iterdir()
if reg_path.suffix.lower() in (".yaml", ".yml")
]

# Shared homeserver config
convert(
"/conf/shared.yaml.j2",
"/conf/workers/shared.yaml",
shared_worker_config=yaml.dump(shared_config),
appservice_registrations=appservice_registrations,
)

# Nginx config
Expand Down