From baba3ac0294929816510ba714179d54f5eb1edde Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 11 Oct 2022 18:40:12 -0500 Subject: [PATCH 01/10] MSC so appservices only interested in local users Fix https://github.com/matrix-org/matrix-spec/issues/1272 --- ...pservice-only-interested-in-local-users.md | 63 +++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 proposals/0000-appservice-only-interested-in-local-users.md diff --git a/proposals/0000-appservice-only-interested-in-local-users.md b/proposals/0000-appservice-only-interested-in-local-users.md new file mode 100644 index 00000000000..9963e8cffdd --- /dev/null +++ b/proposals/0000-appservice-only-interested-in-local-users.md @@ -0,0 +1,63 @@ +# MSC0000: Application Service should only be interested in local users + +Application services receive events that they are "interested" in. + +The current language in the spec describes it like this: + +> An application service is said to be "interested" in a given event if one of the IDs +> in the event match the regular expression provided by the application service +> [registration]. [...] the application service is said to be interested in a given +> event if one of the application service's namespaced users is the target of the event, +> or is a joined member of the room where the event occurred. + +This language is ambiguous around which users it should match against though. Should it +be all members of the room including remote users from other homeservers or just the +local users where the application service lives? It could be assumed either way and +naively applied to all members of the room which is still valid with the current +language and even what Synapse does. + +But matching against remote users is merely a footgun because an application service may +assume that it'll receive all events sent by that remote user, even though it will only +receive events in rooms that are shared with a local user. This leaves us with a +behaivor mismatch between remote and local users. + + + + +## Proposal + +Therefore the proposal is that the `users` namespace regex should only be applied +against local users of the homeserver. + + + +## Potential issues + +There are use cases like moderation where an application service wants to hear all +messages from remote users in rooms but these are are also covered by the `rooms` +namespace where all events in a matched room are considered "interesting". + + + +## Alternatives + +The alternative is clarify that the `users` namespace should be matched against all +users (local and remote). This still leaves us with the behavior-difference footgun. + + + +## Security considerations + +Since we're reducing the surface area, there doesn't seem to be any additional security +considerations introduced. + +With this MSC, an application service will be receiving less events than before. + + + +## Unstable prefix + +If a homeserver wants to implement this functionality before this MSC merges, since it +only affects application services local to the server, it can be implemented although it +will be technically unspecced behavior until this MSC is merged. Also consider a config +option for maximal compatibility with existing application services people may be using. From a1ad5348da33c9682c33943e97dc278260b6a762 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 11 Oct 2022 22:31:47 -0500 Subject: [PATCH 02/10] Add MSC number --- ...ers.md => 3905-appservice-only-interested-in-local-users.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{0000-appservice-only-interested-in-local-users.md => 3905-appservice-only-interested-in-local-users.md} (97%) diff --git a/proposals/0000-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md similarity index 97% rename from proposals/0000-appservice-only-interested-in-local-users.md rename to proposals/3905-appservice-only-interested-in-local-users.md index 9963e8cffdd..5b4a4953a33 100644 --- a/proposals/0000-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -1,4 +1,4 @@ -# MSC0000: Application Service should only be interested in local users +# MSC3905: Application Service should only be interested in local users Application services receive events that they are "interested" in. From 48684a675cd023e40eb85fd17e569727c30bfcae Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 11 Oct 2022 22:33:24 -0500 Subject: [PATCH 03/10] Fix typo --- proposals/3905-appservice-only-interested-in-local-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index 5b4a4953a33..fb311f28206 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -19,7 +19,7 @@ language and even what Synapse does. But matching against remote users is merely a footgun because an application service may assume that it'll receive all events sent by that remote user, even though it will only receive events in rooms that are shared with a local user. This leaves us with a -behaivor mismatch between remote and local users. +behavior mismatch between remote and local users. From 60f3232178eba8c58d8de4d7bdc650c075ee3b4d Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 12 Oct 2022 02:28:31 -0500 Subject: [PATCH 04/10] Fix title --- proposals/3905-appservice-only-interested-in-local-users.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index fb311f28206..818c79ee5b0 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -1,4 +1,4 @@ -# MSC3905: Application Service should only be interested in local users +# MSC3905: Application services should only be interested in local users Application services receive events that they are "interested" in. From b36dee4f913170713a60e8966afb8ae71949ca86 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 12 Oct 2022 02:36:54 -0500 Subject: [PATCH 05/10] Add example implementation See https://github.com/matrix-org/matrix-spec-proposals/pull/3905#discussion_r993004036 --- .../3905-appservice-only-interested-in-local-users.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index 818c79ee5b0..f572394dc42 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -29,6 +29,17 @@ behavior mismatch between remote and local users. Therefore the proposal is that the `users` namespace regex should only be applied against local users of the homeserver. +A basic implementation of this would look like: + +```js +const isLocalUser = sender.endsWith(":" + homeserver.domain); +const isInterestingUser = isLocalUser && sender.matches(regex); +``` + +```js +const localRoomMembers = getLocalRoomMembers(roomId); +const interestingUsers = localRoomMembers.filter((localRoomMember) => localRoomMember.matches(regex)); +``` ## Potential issues From 74f750ca9649ab8429dd8a18f68f295e8d733c50 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 12 Oct 2022 02:38:52 -0500 Subject: [PATCH 06/10] `rooms` and `aliases` not affected See https://github.com/matrix-org/matrix-spec-proposals/pull/3905#discussion_r993003476 --- proposals/3905-appservice-only-interested-in-local-users.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index f572394dc42..6912a5da884 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -41,6 +41,12 @@ const localRoomMembers = getLocalRoomMembers(roomId); const interestingUsers = localRoomMembers.filter((localRoomMember) => localRoomMember.matches(regex)); ``` +--- + +To avoid confusion, please note that the `rooms` and `aliases` namesapces are not +affected. You can still match whatever rooms and aliases to listen to all events +that occur in them. + ## Potential issues From 7b5b8ad2855cbd1931a83d33af0d7e850097a64b Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 12 Oct 2022 02:51:56 -0500 Subject: [PATCH 07/10] Add historical context See https://github.com/matrix-org/matrix-spec-proposals/pull/3905#discussion_r993005189 --- ...pservice-only-interested-in-local-users.md | 20 +++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index 6912a5da884..bebeaa43975 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -22,8 +22,6 @@ receive events in rooms that are shared with a local user. This leaves us with a behavior mismatch between remote and local users. - - ## Proposal Therefore the proposal is that the `users` namespace regex should only be applied @@ -71,6 +69,24 @@ considerations introduced. With this MSC, an application service will be receiving less events than before. +## Historical context + +According to @turt2live (SCT member), "the spec intended to [originally] say the +namespace can make an appservice interested in remote users, though there's obviously no +ability for the server to call `/user` on remote users (it's not like the appservice can +create them)." (https://github.com/matrix-org/synapse/pull/13958#discussion_r988369446) + +This intention goes back further than `r0` (or `v1.0` in marketing versions speak) but +this history is lost to time since there isn't really anything concrete to point to +beyond the original spec +[issue](https://github.com/matrix-org/matrix-spec-proposals/issues/1307) and +[PR](https://github.com/matrix-org/matrix-spec-proposals/pull/1533) which don't mention +these details. + +Since we're unable to come up with any valid use cases nowadays, it's unclear to +outsiders from that time whether the original intention is actually true. In any case, +we're clarifying it here and making an MSC to change it explicitly. + ## Unstable prefix From a280c247a1bb511dce413a7834e032418803ff67 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 12 Oct 2022 02:53:26 -0500 Subject: [PATCH 08/10] Document how to do unstable See https://github.com/matrix-org/matrix-spec-proposals/pull/3905#discussion_r992960361 --- .../3905-appservice-only-interested-in-local-users.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index bebeaa43975..b545df1eb36 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -90,7 +90,9 @@ we're clarifying it here and making an MSC to change it explicitly. ## Unstable prefix -If a homeserver wants to implement this functionality before this MSC merges, since it -only affects application services local to the server, it can be implemented although it -will be technically unspecced behavior until this MSC is merged. Also consider a config -option for maximal compatibility with existing application services people may be using. +While this MSC is not considered stable, appservices can opt-in to this behaviour in +either of two ways: + + 1. Specifying a regex which includes the local homeserver's domain in the regex (eg: + `@_irc.*:example.org`) + 1. Adding `"org.matrix.msc3905": true` to the root level of their registration file From 0b9fdb0fc7728fb85bec0f0cb478e83b9643c534 Mon Sep 17 00:00:00 2001 From: Andrew Morgan Date: Thu, 13 Oct 2022 15:37:02 +0000 Subject: [PATCH 09/10] fix some typos --- proposals/3905-appservice-only-interested-in-local-users.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index b545df1eb36..613e2775096 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -41,7 +41,7 @@ const interestingUsers = localRoomMembers.filter((localRoomMember) => localRoomM --- -To avoid confusion, please note that the `rooms` and `aliases` namesapces are not +To avoid confusion, please note that the `rooms` and `aliases` namespaces are not affected. You can still match whatever rooms and aliases to listen to all events that occur in them. @@ -49,14 +49,14 @@ that occur in them. ## Potential issues There are use cases like moderation where an application service wants to hear all -messages from remote users in rooms but these are are also covered by the `rooms` +messages from remote users in rooms but these are also covered by the `rooms` namespace where all events in a matched room are considered "interesting". ## Alternatives -The alternative is clarify that the `users` namespace should be matched against all +The alternative is to clarify that the `users` namespace should be matched against all users (local and remote). This still leaves us with the behavior-difference footgun. From dbe5bde6b220206dbb33ddf217b2196d7ab06598 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Wed, 19 Oct 2022 12:06:19 -0500 Subject: [PATCH 10/10] Add link to spec Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3905-appservice-only-interested-in-local-users.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/3905-appservice-only-interested-in-local-users.md b/proposals/3905-appservice-only-interested-in-local-users.md index 613e2775096..d78df3903f5 100644 --- a/proposals/3905-appservice-only-interested-in-local-users.md +++ b/proposals/3905-appservice-only-interested-in-local-users.md @@ -2,7 +2,8 @@ Application services receive events that they are "interested" in. -The current language in the spec describes it like this: +The [current language in the spec](https://spec.matrix.org/v1.4/application-service-api/#registration) +describes it like this: > An application service is said to be "interested" in a given event if one of the IDs > in the event match the regular expression provided by the application service