Format for rich text messages (SPEC-48)

[matrixbot]

Currently, m.text just supports a raw string. Whilst we have solved the attachment use case (via m.image, m.video and so on), we have no current way to specify markup for raw text.

After doing some research, there are a handful of options here:

• Roll our own format e.g.

  spans: [ 
    { type: bold, start: 4, end: 9 }, 
    { type: italic, start: 15, end: 21 } 
  ]

• Use XHTML tags (and strip the nasty ones)
• Use BBCode
• Use Markdown
• Use reStructured Text (RST)

I would prefer that we didn't re-invent the wheel on this, so I'm against rolling our own format. Using XHTML tags would be fine, but the kicker is that there are unsafe tags which we'd want to strip (akin to XEP-0071). Clients may be lazy and not strip bad tags, and everything goes into a mess. For these reasons, I'm against using XHTML as the markup for this.

This leave BBCode and Markdown/RST. BBCode has been around for a long time, has easy regex parsing to map directly to HTML, though there are odd quirks with it such as ambiguous markup syntax. Markdown/RST have a lot of libraries for different languages to map to HTML, and provide a richer feature set than BBCode. As such, I'd prefer either Markdown or RST.

Without getting into the holy war of RST vs Markdown, RST is more mature at the moment since there is a well-defined spec http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html and not many different fragmented versions (unlike Markdown). That being said, this will be fixed in the future http://commonmark.org/ - I hold no real preference as to which we should use. It seems Markdown has a slightly nicer syntax and is geared more towards generating HTML possibly, which I feel may be more aligned with what we actually want from a markup language.

Thoughts welcome.

(Imported from https://matrix.org/jira/browse/SPEC-48)

(Reported by @kegsay)

[matrixbot]

Jira watchers: @erikjohnston @kegsay @leonerd @ara4n @richvdh

[matrixbot]

As for the message types, I would propose:

• m.text.markdown
• m.emote.markdown
• m.notice.markdown

We could be pedantic and go with m.text.markdown.commonmark, but not sure how much that will gain us. If we end up using reStructured Text instead, then m.text.rst etc. Given RST has a spec, this doesn't need another namespace beneath it.

-- @kegsay

[matrixbot]

I'm not sure I want to conflate the type of the message and the way it is formatted, i.e., it seems a bit weird to have "m.text" and "m.text.markdown" (or "m.text.plain" and "m.text.markdown".) Whether or not it's an emote/notice and how it is formatted seem to be fairly orthogonal.

-- @erikjohnston

[matrixbot]

Also, we need to consider fallback support for clients that don't understand whatever format we decide on. I guess things like format are human readable enough that it is probably fine to just output the raw content.

-- @erikjohnston

[matrixbot]

So you would propose keeping m.text and then what? Having a body_markdown key or something? That could get old pretty quick, since you'd still need to specify the body key for fallback, resulting in unnecessarily duplicated information.

-- @kegsay

[matrixbot]

Or you could have a body_type: markdown, which could be reused across different events.

-- @erikjohnston

[matrixbot]

body_types sound good to me, as long as we carefully separate the plain-text representation from whatever whacky rich-text representation we go for, so the event can be interpreted as a normal m.text and still work.

In terms of the representation to use - i think we need to support the richest common language, so as to interop as easily as possible with other platforms (after all, Matrix is intended as glue between platforms). And that means HTML. Agreed that it's a security nightmare, but if we start with a truly minimal subset and only add things in very conservatively, I suspect it could be okay? Agreed it's a faff for the client writer who has to strip out the nasty stuff though, but it's not that much worse than programming against any other XSS exploit... More of a problem is how to actually style it - having foo a la AIM is EVIL, and having foo isn't much better. We could deny CSS entirely? Or provide a set of CSS classes we mandate must be supported?

Or we recommend markdown by default, and provide HTML for compat? For the record I've really not enjoyed writing RST, whereas markdown generally feels quite intuitive.

-- @ara4n

[matrixbot]

My concern with HTML is that its probably quite fiddly if you're not planning to dump it straight into a web view but still have some scope for formatting. I'm inclined to favour whatever is easier for clients and then let the bridges have the burden of reformatting as required.

It's worth noting that I don't think markdown allows changing of fonts or colours.

-- @erikjohnston

[matrixbot]

Other thoughts in favour of HTML:

• We want copy & paste to work on browsers, which means supporting HTML
• We get nice graphical HTML authoring functionality for free nowadays in browsers

Interesting that Lync separately supports RTF, HTML (and Ink) rich message payloads: http://blogs.technet.com/b/lync/archive/2009/03/04/rich-content-and-html-in-instant-messages.aspx

We absolutely do not want gateways to have to do crappy impedance mismatched conversions between these formats, but pass them through where we can - with instructions for clients to either show the plain-text equivalent, or do whatever security voodoo is required to show it safely. I think the most useful thing the gateway could do is to convert it into our nice-rich-intermediary-common-language (i.e. HTML),whilst keeping the original around too for those who can understand it.

Erik: i don't understand "its probably quite fiddly if you're not planning to dump it straight into a web view but still have some scope for formatting".

-- @ara4n

[matrixbot]

I've just sat and skimmed over the CommonMark spec. There's nothing in there to support such formatting as colours, which I suspect were one of the original intents of this idea. It supports <em> and <strong>, but no other character-wise formatting (underlines? strikethrough? other fonts?). It seems that CommonMark, and other other Markdown-derived formats are all about high-level macro-structuring and not about the low-level character-formatting of text.

If we want to go down the HTML-like route, could it not make sense for homeservers to parse the incoming text, check all the tags against a whitelist and reject bad ones? Whatever's left it can presumbly just strip out automatically and come up with a plain-text body itself, can't it?

-- @leonerd

[matrixbot]

Agreed that markdown isn't rich enough (although I really like the idea of supporting it as an optional body type).

We can certainly mandate that HS enforces HTML validation rules though to avoid pain for clients - it's equivalent to a web forum engine going and stripping off malicious HTML serverside before serving it up. I also suggest that sanitizing HTML really is a solved problem these days - every single HTML-backed authoring system out there (Wordpress & co) has had to solve it. We should be able to borrow an existing one (so long as it explicitly whitelists valid tags & attributes & valid attribute value grammars (e.g. valid URI schemes); there were some embarassingly sloppy ones back in the day that tried to strip invalid content rather than explicitly whitelist valid content).

The biggest challenge I see is to decide what level of styling to support. Arbitrary CSS is a huge security hole too. But predefined CSS class names is pretty clunky. Again, I assume someone has solved this well - perhaps similarly providing a secure subset of CSS which can be used without risk of crashing browsers or doing cross-site attacks or abusing the rest of the page etc... In fact, isn't this the problem that every RSS aggregator ever has had to solve?

Example approaches:

• http://codex.wordpress.org/Function_Reference/wp_kses
• http://jsoup.org/cookbook/cleaning-html/whitelist-sanitizer
• https://www.npmjs.org/package/sanitize-html
• https://github.com/rgrove/sanitize
• https://pythonhosted.org/feedparser/html-sanitization.html
• https://github.com/jsocol/bleach

Given Synapse is in pythonland, I suggest that bleach would be a good bet and seems widely respected. But we'd need to obviously spell out the whitelist of tags, attributes & their allowed values etc in the spec.

-- @ara4n

[matrixbot]

The more I think about this, the more dubious I am that the HS should be validating the messages. Reasons are:

• Impossible with E2E encryption
• The client may want to apply further sanitation rules based on what its HTML renderer can actually handle
• Forces the HS to keep a sanitised copy of messages around alongside the 'official' one, which is a bit messy, and the sanitised copy's crypto signature will obviously be broken.

So my hunch is that "it's up to clients to sanity-check and escape untrusted content that they display - and untrusted content means stuff received from the rest of Matrix. We provide SDKs that do this already to make it easier for client implementors.". So we could use something like google-caja in JS to fix up the display (and cache it) on the client as required.

Obviously the big downside on this is that it increases the burden on client implementers. But in practice, rich text is a relatively exotic feature - is this actually such a big problem?

-- @ara4n

[matrixbot]

Final thought: we have to actually consider the use cases here:

1. Kegan wants his bot to emit spans of bold red text
2. Matthew wants the option of sometimes saying CRAP and for it to come out bold. I'd want it to be up to the client to give me the best UI for bolding the text (eg iOS could use the normal rich text styling; commandline client might do something markdowny, etc)
3. Amandine wants to copy hunks of formatted content from Chrome or Excel and paste them via a rich-text input field in Matrix to Matthew and have them come out looking vaguely right.
4. Gary wants to copy hunks of RTF out of powerpoint and send it to Matthew via Lync, which Matthew wants to receive with the intended formatting in Matrix.
5. ...i'm sure there are others.

3 and 4 mandate we support some level of flexible styling. 3 also implies we might need to do RTF->HTML conversions somewhere (ew) given both Windows and OSX are prone to using RTF on their clipboard by default, with HTML being a rather broken second class citizen. Clearly we need to specify a single featureful common language for the rich markup though, and given Matrix is all about being web-friendly and built-for-the-web, i guess HTML5 is it.

-- @ara4n

[matrixbot]

I really dislike where this is headed: I would rather not have to rely on clients to sanitise HTML :/ We have the opportunity to remove an entire attack vector here, I think we should use any other form of markup language. It's all nice and well whitelisting tags, but it means jack all when lazy clients don't bother, and as far as I was aware, we want to make client implementations as easy as possible. I think using HTML is a huge design fail, because when a dev quickly hacks rich text support into their client and everything works fine, they dust their hands and call it a day, and can be completely unaware of the security hole they've just developed. We can do better than this.

As for the use cases: #1 and #2 can really be done by anything. #3: How exactly is this being copied? I'm not clear on how this actually works. #4: RTF > HTML > Some markup language would work, though I appreciate the indirection is mingy.

I don't think saying "well we will just provide an SDK which will do the sanitisation" is a sufficiently good enough reason to use HTML. People can easily write a noddy "send message from A to B" client, and if rich text is in HTML, it's _trivial_ for someone to think "oh okay, I'll just wodge that in quickly". It's too easy to get wrong I think.

-- @kegsay

[matrixbot]

If you can show me a markup language as expressive as HTML without XSS vulnerabilities, which has parity for converting stuff in & out of HTML (for displaying in webbrowsers and parsing HTML and RTF clipboard contents) then let's use it. But I really don't want to cripple our rich text support just because we can't trust client developers to avoid XSS attacks. Matrix is about improving user experience in the end - not being penalised by developmental quirks.

-- @ara4n