The search endpoint does not mention the list of allowed values for `room_events.keys`

[babolivier]

Link to problem area: https://spec.matrix.org/v1.3/client-server-api/#post_matrixclientv3search

Issue

The documentation for the room_events.keys parameter does not mention a restricted list of keys to use for search, implying (as per my own interpretation) that all keys in an event are searchable. However, since matrix-org/synapse#307 (which predates the spec for this endpoint) Synapse has always restricted this list to ["content.body", "content.name", "content.topic"].

Expected behaviour

The spec should mention that the values for room_events.keys must be included in ["content.body", "content.name", "content.topic"].

[babolivier]

As an aside, this got me fairly confused because the spec currently says "Defaults to all" for this parameter (which to me implies all possible keys in an event) but Synapse defaults to the aforementioned list.

[anoadragon453]

(which predates the spec for this endpoint)

From the linked PR, it looks like both the spec wording and Synapse PR were implemented at the same time:

• Synapse PR
• Spec PR

And at the time, formatted_body didn't exist: https://github.com/matrix-org/matrix-spec-proposals/blob/716c5b7a8b6d4a4109e0fee16929a88fae04bda4/event-schemas/schema/v1/m.room.message. In fact, it appears that formatted_body didn't make it to the spec until 3 years later.

So I would argue that Synapse ideally would have added searching in formatted_body once that field was added to the spec.

[babolivier]

From the linked PR, it looks like both the spec wording and Synapse PR were implemented at the same time

Right, yes, roughly. I just looked at the dates the PRs were merged at and saw the Synapse one merged before the spec one, but it's a 4d difference.

So I would argue that Synapse ideally would have added searching in formatted_body once that field was added to the spec.

To be clear, I'm not saying that shouldn't have happened (and ideally we would have a way to search for all keys in an event). This issue is about the fact that currently the spec doesn't represent the way Matrix has been working in practice with regards to that feature for as long as it's existed.

Also, I'd argue that even if Synapse supported searching in formatted_body, the spec would still be at best confusing and at worse wrong by omitting a list of allowed keys. I would interpret that as the fact that I can search in content.msgtype, content.redacts, sender, etc., which still wouldn't reflect reality.

[turt2live]

From looking at the original implementation and spec, it appears as though the intent was to support a generic search interface which defaults to a set of 3 keys ("all" which were feasible at the time). With this in mind, I would argue that it'd be a Synapse bug that fields are not searched today when they should be, and the Spec needs a clarification to explain how the dot notation [was meant to] work.

An MSC would be required to change the format of the search, as would changing what "all" means.