|
| 1 | +# MSC2781: Remove reply fallbacks from the specification |
| 2 | + |
| 3 | +Currently the specification suggests clients should send and strip a |
| 4 | +[fallback representation](https://spec.matrix.org/v1.10/client-server-api/#fallbacks-for-rich-replies) |
| 5 | +of a replied to message. The fallback representation was meant to simplify |
| 6 | +supporting replies in a new client, but in practice they add complexity, are |
| 7 | +often implemented incorrectly and block new features. |
| 8 | + |
| 9 | +This MSC proposes to **remove** those fallbacks from the specification. |
| 10 | + |
| 11 | +Some of the known issues include: |
| 12 | +* The content of reply fallback is [untrusted](https://spec.matrix.org/v1.10/client-server-api/#stripping-the-fallback). |
| 13 | +* Reply fallbacks may leak history. ([#368](https://github.com/matrix-org/matrix-spec/issues/368)) |
| 14 | +* Parsing reply fallbacks can be tricky. ([#350](https://github.com/matrix-org/matrix-spec/issues/350)) |
| 15 | +* It is unclear how to handle a reply to a reply. ([#372](https://github.com/matrix-org/matrix-spec/issues/372)) |
| 16 | +* Localization of replies is not possible when the content is embedded into the event. |
| 17 | +* It is not possible to fully redact an event once it is replied to. This causes issues with Trust & Safety where |
| 18 | + spam or other removed content remains visible, and may cause issues with the GDPR Right to be Forgotten. |
| 19 | +* There are a variety of implementation bugs related to reply fallback handling. |
| 20 | + |
| 21 | +More details and considerations are provided in the appendices, but these are |
| 22 | +provided for convenience and aren't necessary to understand this proposal. |
| 23 | + |
| 24 | +## Proposal |
| 25 | + |
| 26 | +Remove the [rich reply fallback from the |
| 27 | +specification](https://spec.matrix.org/v1.10/client-server-api/#fallbacks-for-rich-replies). |
| 28 | +Clients should stop sending them and should consider treating `<mx-reply>` parts |
| 29 | +as they treat other invalid html tags. |
| 30 | + |
| 31 | +Clients are not required to include a fallback in a reply since version 1.3 of |
| 32 | +the |
| 33 | +[specification](https://spec.matrix.org/v1.10/client-server-api/#rich-replies). |
| 34 | +For this reason the reply fallback can be removed from the specification without |
| 35 | +any additional deprecation period. |
| 36 | + |
| 37 | +A suggestion for the spec PR: An info box could be included to mention |
| 38 | +the historical use of the reply fallback, suggesting that clients may encounter |
| 39 | +such events sent by other clients and that clients may need to strip out such |
| 40 | +fallbacks. |
| 41 | + |
| 42 | +Given clients have had enough time to implement replies completely, the |
| 43 | +overall look & feel of replies should be unchanged or even improved by this |
| 44 | +proposal. Implementing replies in a client should also be a bit easier with this |
| 45 | +change. |
| 46 | + |
| 47 | +An extended motivation is provided at [the end of this document](#user-content-appendix-b-issues-with-the-current-fallbacks). |
| 48 | + |
| 49 | +## Potential issues |
| 50 | + |
| 51 | +Old events and events sent by clients implementing an older version of the |
| 52 | +Matrix specification might still contain a reply fallback. So for at least some |
| 53 | +period of time clients will still need to strip reply fallbacks from messages. |
| 54 | + |
| 55 | +Clients which don't implement rich replies may see messages without context, |
| 56 | +confusing users. However, most replies are in close proximity to the original |
| 57 | +message, making context likely to be nearby. Clients should also have enough |
| 58 | +information in the event to render helpful indications to users while they work |
| 59 | +on full support. |
| 60 | + |
| 61 | +Clients which aren't using |
| 62 | +[intentional mentions](https://spec.matrix.org/v1.7/client-server-api/#mentioning-the-replied-to-user) |
| 63 | +may cause some missed notifications on the receiving side. |
| 64 | +[MSC3664](https://github.com/matrix-org/matrix-doc/pull/3664) and similar aim to |
| 65 | +address this issue, and |
| 66 | +[MSC4142](https://github.com/matrix-org/matrix-spec-proposals/pull/4142) tries |
| 67 | +to improve the intentional mentions experience for replies generally. |
| 68 | +Because intentional mentions are already part of the Matrix specification since |
| 69 | +version 1.7, clients can be expected to implement those first, which should make |
| 70 | +the impact on notifications minimal in practice. |
| 71 | + |
| 72 | +## Alternatives |
| 73 | + |
| 74 | +[MSC2589](https://github.com/matrix-org/matrix-doc/pull/2589): This adds the |
| 75 | +reply text as an additional key. While this solves the parsing issues, it |
| 76 | +doesn't address the other issues with fallbacks. |
| 77 | + |
| 78 | +One could also just stick with the current fallbacks and make all clients pay |
| 79 | +the cost for a small number of clients actually benefitting from them. |
| 80 | + |
| 81 | +Lastly one could introduce an alternative relation type for replies without |
| 82 | +fallback and deprecate the current relation type (since it does not fit the |
| 83 | +[new format for relations](https://github.com/matrix-org/matrix-doc/pull/2674) |
| 84 | +anyway). We could specify, that the server is supposed to send the replied_to |
| 85 | +event in unsigned to the client, so that clients just need to stitch those two |
| 86 | +events together, but don't need to fetch the replied_to event from the server. |
| 87 | +It would make replies slightly harder to implement for clients, but it would be |
| 88 | +simpler than what this MSC proposes. |
| 89 | + |
| 90 | +## Security considerations |
| 91 | + |
| 92 | +Overall this should **reduce** security issues as the handling of untrusted |
| 93 | +HTML is simplified. For an example security issue that could be avoided, see |
| 94 | +https://github.com/vector-im/element-web/releases/tag/v1.7.3 and the appendix. |
| 95 | + |
| 96 | +## Unstable prefix |
| 97 | + |
| 98 | +No unstable prefix should be necessary as clients aren't required to send reply |
| 99 | +fallbacks for all messages since version 1.3 of the Matrix specification, which |
| 100 | +changed the wording from "MUST" to "SHOULD". |
| 101 | + |
| 102 | +## Appendix A: Support for rich replies in different clients |
| 103 | + |
| 104 | +### Clients without rendering support for rich replies |
| 105 | + |
| 106 | +Of the 23 clients listed in the [Matrix client matrix](https://matrix.org/clients-matrix) |
| 107 | +16 are listed as not supporting replies (updated January 2022): |
| 108 | + |
| 109 | +- Element Android: Relies on the reply fallback. |
| 110 | +- Element iOS: [Does not support rich replies](https://github.com/vector-im/element-ios/issues/3517) |
| 111 | +- weechat-matrix: Actually has an [implementation](https://github.com/poljar/weechat-matrix/issues/86) to send replies although it seems to be [broken](https://github.com/poljar/weechat-matrix/issues/233). Doesn't render rich replies. Hard to implement because of the single socket implementation, but may be easier in the Rust version. |
| 112 | +- Quaternion: [Blocked because of fallbacks](https://github.com/quotient-im/libQuotient/issues/245). |
| 113 | +- matrixcli: [Doesn't support formatted messages](https://github.com/ahmedsaadxyzz/matrixcli/issues/10). |
| 114 | +- Ditto Chat: [Seems to rely on the fallback](https://gitlab.com/ditto-chat/ditto/-/blob/main/mobile/scenes/chat/components/Html.tsx#L38) |
| 115 | +- Mirage: Supports rich replies, but [doesn't strip the fallback correctly](https://github.com/mirukana/mirage/issues/89) and uses the fallback to render them. |
| 116 | +- Nio: [Unsupported](https://github.com/niochat/nio/issues/85). |
| 117 | +- Pattle: Client is not being developed anymore. |
| 118 | +- Seaglass: Doesn't support rich replies, but is [unhappy with how the fallback looks](https://github.com/neilalexander/seaglass/issues/51)? |
| 119 | +- Miitrix: Somewhat unlikely to support it, I guess? |
| 120 | +- matrix-commander: No idea, but doesn't look like it. |
| 121 | +- gotktrix: [Seems to rely on the reply fallback](https://github.com/diamondburned/gotktrix/blob/5f2783d633560421746a82aab71d4f7421e4b99c/internal/app/messageview/message/mcontent/text/html.go#L437) |
| 122 | +- Hydrogen: [Seems to use the reply fallback](https://github.com/vector-im/hydrogen-web/blob/c3177b06bf9f760aac2bfd5039342422b7ec8bb4/doc/impl-thoughts/PENDING_REPLIES.md) |
| 123 | +- kazv: Doesn't seem to support replies at all |
| 124 | +- Syphon: [Uses the reply fallback in body](https://github.com/syphon-org/syphon/blob/fa44c5abe37bdd256a9cb61cbc8552e0e539cdce/lib/views/widgets/messages/message.dart#L368) |
| 125 | + |
| 126 | +So in summary, 3/4 of the listed clients don't support replies. At least one |
| 127 | +client doesn't support it because of the fallback (Quaternion). 3 of the command |
| 128 | +line clients probably won't support replies, since they don't support formatted |
| 129 | +messages and replies require html support for at least sending. |
| 130 | + |
| 131 | +Only one client implemented rich replies in the last 1.5 years after the |
| 132 | +original list was done in October 2020. Other clients are either new in my list |
| 133 | +or didn't change their reply rendering. I would appreciate to hear, why those |
| 134 | +client developers decided not to support rich reply rendering and if dropping |
| 135 | +the reply fallback would be an issue for them. |
| 136 | + |
| 137 | +Changes from 1.5 years ago as of January 2022: |
| 138 | + |
| 139 | +- Fractal: [Seems to support replies now!](https://gitlab.gnome.org/GNOME/fractal/-/merge_requests/941) |
| 140 | +- Commune: Seems to support rich reply rendering and style them very nicely. |
| 141 | +- NeoChat: [Supports rich replies](https://invent.kde.org/network/neochat/-/blob/master/src/utils.h#L21) |
| 142 | +- Cinny: [Seems to support rich replies](https://github.com/ajbura/cinny/blob/6ff339b552e242f6233abd86768bb2373b150f77/src/app/molecules/message/Message.jsx#L111) |
| 143 | +- gomuks: [Strips the reply fallback](https://github.com/tulir/gomuks/blob/3510d223b2d765572bf2e97222f2f55d099119f0/ui/messages/html/parser.go#L361) |
| 144 | +- Lots of other new clients! |
| 145 | + |
| 146 | + |
| 147 | +### Results of testing replies without fallback |
| 148 | + |
| 149 | +So far I haven't found a client that completely breaks without the fallback. |
| 150 | +All clients that support rendering rich replies don't break, when there is no |
| 151 | +fallback according to my tests (at least Nheko, Element/Web, FluffyChat and |
| 152 | +NeoChat were tested and some events without fallback are in #nheko:nheko.im and |
| 153 | +I haven't heard of any breakage). Those clients just show the reply as normal |
| 154 | +and otherwise seem to work completely fine as well. Element Android and Element |
| 155 | +iOS just don't show what message was replied to. Other clients haven't been |
| 156 | +tested by the author, but since the `content` of an event is untrusted, a client |
| 157 | +should not break if there is no reply fallback. Otherwise this would be a |
| 158 | +trivial abuse vector. |
| 159 | + |
| 160 | + |
| 161 | +## Appendix B: Issues with the current fallbacks |
| 162 | + |
| 163 | +This section was moved to the back of this MSC, because it is fairly long and |
| 164 | +exhaustive. It lists all the issues the proposal author personally experienced |
| 165 | +with fallbacks in their client and its interactions with the ecosystem. |
| 166 | + |
| 167 | +### Stripping the fallback |
| 168 | + |
| 169 | +To reply to a reply, a client needs to strip the existing fallback of the first |
| 170 | +reply. Otherwise replies will just infinitely nest replies. |
| 171 | +[While the spec doesn't necessarily require stripping the fallback in replies to replies (only for rendering)](https://spec.matrix.org/v1.1/client-server-api/#fallback-for-mtext-mnotice-and-unrecognised-message-types), |
| 172 | +not doing so risks running into the event size limit, but more importantly, it |
| 173 | +just leads to a bad experience for clients actually relying on the fallback. |
| 174 | + |
| 175 | +Stripping the fallback is not trivial. Multiple implementations had bugs in |
| 176 | +their fallback stripping logic. The edge cases are not covered in the |
| 177 | +specification in detail and some clients have interpreted them differently. |
| 178 | +Common mistakes include: |
| 179 | + |
| 180 | +- Not stripping the fallback in body, which leads to a very long nested chain. |
| 181 | +- Not dealing with mismatched `<mx-reply>` tags, which can look like you were |
| 182 | + impersonating someone. |
| 183 | + |
| 184 | +For the `body` extra attention needs to be paid to only strip lines starting |
| 185 | +with `>` until the first empty line. Implementations either only stripped the |
| 186 | +first line, stripped all lines starting with `>` until the first non empty line, |
| 187 | +that does not start with `>` or stripped only the `formatted_body`. While those |
| 188 | +are implementation bugs, they can't happen if you don't need to strip a |
| 189 | +fallback. |
| 190 | + |
| 191 | +### Creating a new fallback |
| 192 | + |
| 193 | +To create a new fallback, a client needs to add untrusted html to its own |
| 194 | +events. This is an easy attack vector to inject your own content into someone |
| 195 | +elses reply. While this can be prevented with enough care, since Riot basically |
| 196 | +had to fix this issue twice, it can be expected that other clients can also be |
| 197 | +affected by this. |
| 198 | + |
| 199 | +### Requirement of html for replies |
| 200 | + |
| 201 | +The spec requires rich replies to have a fallback using html: |
| 202 | + |
| 203 | +> Rich replies MUST have a format of org.matrix.custom.html and therefore a formatted_body alongside the body and appropriate msgtype. |
| 204 | +
|
| 205 | +This means you can't reply using only a `body` and you can't reply with an |
| 206 | +image, since those don't have a `formatted_body` property currently. This means |
| 207 | +a text only client, that doesn't want to display html, still needs to support |
| 208 | +html anyway and that new features are blocked, because of fallbacks. |
| 209 | + |
| 210 | +### Format is unreliable |
| 211 | + |
| 212 | +While the spec says how a fallback "should" look, there are variations in use |
| 213 | +which further complicates stripping the fallback or are common mistakes, when |
| 214 | +emitting the fallback. Some variations include localizing the fallback, |
| 215 | +missing suggested links or tags, using the body in replies to files or images |
| 216 | +or using the display name instead of the matrix id. |
| 217 | + |
| 218 | +As a result the experience in clients relying on the fallback or stripping the |
| 219 | +fallback varies depending on the sending client. |
| 220 | + |
| 221 | +### Replies leak history |
| 222 | + |
| 223 | +A reply includes the `body` of another event. This means a reply to an event can |
| 224 | +leak data to users, that joined this room at a later point, but shouldn't be |
| 225 | +able to see the event because of visibility rules or encryption. While this |
| 226 | +isn't a big issue, there is still an issue about it: https://github.com/matrix-org/matrix-doc/issues/1654 |
| 227 | + |
| 228 | +This history leak can also cause abusive or redacted messages to remain visible |
| 229 | +to other room members, depending on the client implementation of replies. |
| 230 | + |
| 231 | +Historically clients have also sometimes localized the fallbacks. In those cases |
| 232 | +they leak the users language selection for their client, which may be personal |
| 233 | +information. |
| 234 | + |
| 235 | +### Using the unmodified fallback in clients and bridges |
| 236 | + |
| 237 | +The above issues are minor, if reply fallbacks added sufficient value to |
| 238 | +clients. Bridges usually try to bridge to native replies, so they need to |
| 239 | +strip the reply fallback |
| 240 | +(https://github.com/matrix-org/matrix-doc/issues/1541). Even the IRC bridge |
| 241 | +seems to send a custom fallback, because the default fallback is not that |
| 242 | +welcome to the IRC crowd, although the use cases for simple, text only bridges |
| 243 | +is often touted as a good usecase for the fallback (sometimes even explicitly |
| 244 | +mentioning bridging to IRC). As a result there are very few bridges, that |
| 245 | +benefit from the fallback being present. |
| 246 | + |
| 247 | +Some clients do choose not to implement rich reply rendering, but the experience |
| 248 | +tends to not be ideal, especially in cases where you reply to an image and now |
| 249 | +the user needs to guess, what image was being replied to. |
| 250 | + |
| 251 | +As a result the fallbacks provide value to only a subset of the Matrix |
| 252 | +ecosystem. |
| 253 | + |
| 254 | +### Fallbacks increase integration work with new features |
| 255 | + |
| 256 | +- [Edits explicitly mention](https://github.com/matrix-org/matrix-doc/pull/2676) |
| 257 | + that a reply fallback should not be sent in the `m.new_content`. This causes |
| 258 | + issues for clients relying on the fallback, because they won't show replies |
| 259 | + once a message has been edited (see Element Android as a current example) |
| 260 | + and similar edge cases. |
| 261 | +- [Extensible events](https://github.com/matrix-org/matrix-doc/pull/1767) |
| 262 | + require an update to the specification for fallbacks (because there is no |
| 263 | + `body` or `formatted_body` anymore after the transition period). |
| 264 | + [The current proposal](https://github.com/matrix-org/matrix-doc/pull/3644) |
| 265 | + also intends to just drop the fallbacks in extensible events. |
| 266 | + |
| 267 | +### Localization |
| 268 | + |
| 269 | +Since the fallback is added as normal text into the message, it needs to be |
| 270 | +localized for the receiving party to understand it. This however proves to be a |
| 271 | +challenge, since users may switch languages freely in a room and it is not easy |
| 272 | +to guess, which language was used in a short message. One could also use the |
| 273 | +client's language, but that leaks the user's localization settings, which can be a |
| 274 | +privacy concern and the other party may not speak that language. Alternatively a |
| 275 | +client can just send english fallbacks, but that significantly worsens the |
| 276 | +experience for casual users in non-english speaking countries. The specification |
| 277 | +currently requires them to not be translated (although some clients don't follow |
| 278 | +that), but not sending a fallback at all completely sidesteps the need for the |
| 279 | +spec to specify that and clients relying on an english only fallback. |
0 commit comments