🐟 breaks join rules

[nexy7574]

"allow": "🐟" breaks the join rules in dendrite, and presumably this is caused by GMSL.

Take for example

{"allow": "🐟", "join_rule": "knock"}

this m.room.join_rules event content.

From what I can gather, rather than ignoring the invalid allow field, GMSL is failing to unmarshal the event at all, causing the JoinRuleEventContent.JoinRule to be "", or an empty string, which is not allowed by spec, causing valid joins to fail.

The solution to this is to ignore the invalid allow value (unless the join_rule is restricted or knock_restricted, otherwise the event must always fail joins without an invite per the spec), however I dug into the code briefly myself and got a bit lost. It's entirely possible that it's not an unmarshal error that's being discarded but in fact just the join rule itself being entirely discarded and consequently not set, leading to the default for string being used.

[kegsay]

It's not as simple as failing to unmarshal - https://go.dev/play/p/v-9Cl3v7mdU unmarshals just fine. When used as part of an event, the JSON is canonicalised, which has historically has issues with non UTF-8 characters, so potentially that? Without an error message it's hard to debug.

[nexy7574]

This actually looks tricker than I first realised. Upon further inspection, I cannot actually repro with a fresh room with just the join rule I sent above, nor with some other tricks.

After taking a stab at some of the auth event IDs in the relevant logs (attached) it looks like my fish may be in another castle. The room dendrite was struggling to join had some events sent by a user ID with UTF-8 characters in their user ID, and it looks like they've managed to end up in the auth chain (also attached).

If you need any of the events included in the log, let me know (or ping me on Matrix) and I'll happily supply them.

log-sample.txt
auth-chain.txt

[nexy7574]

Can confirm this is actually more likely caused by a user with an invalid user ID sending an auth event used to join, which gmsl consequently cannot parse, failing:

time="2025-04-05T01:37:09.912637143Z" level=info msg="User requested to room join" room_id="#the-fish-room:nexy7574.co.uk" servers="[]" user_id="@nex:dendrite.nexy7574.co.uk"
time="2025-04-05T01:37:09.995957113Z" level=info msg="Checking event signatures for 37 events of room state" context=missing
time="2025-04-05T01:37:10.009665595Z" level=warning msg="Signature validation failed for event \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\"" error="invalid sender userID: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.009894847Z" level=warning msg="Signature validation failed for event \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\"" error="invalid sender userID: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.010054018Z" level=warning msg="Signature validation failed for event \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\"" error="invalid sender userID: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.010267368Z" level=warning msg="Signature validation failed for event \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\"" error="invalid sender userID: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.012601688Z" level=warning msg="Event \"$mZwQzOz0RbOGs-2ZwVQ9wv4zCaGHkAAg7bwsn1L94oE\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$mZwQzOz0RbOGs-2ZwVQ9wv4zCaGHkAAg7bwsn1L94oE\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.017540978Z" level=warning msg="Event \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.017872196Z" level=warning msg="Event \"$pAn0iqAyn87gmnU7QSkxndcPEvAOz48gI8IuL3wRV6w\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$pAn0iqAyn87gmnU7QSkxndcPEvAOz48gI8IuL3wRV6w\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.017944689Z" level=warning msg="Event \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.018250672Z" level=warning msg="Event \"$pAn0iqAyn87gmnU7QSkxndcPEvAOz48gI8IuL3wRV6w\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$pAn0iqAyn87gmnU7QSkxndcPEvAOz48gI8IuL3wRV6w\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.018300092Z" level=warning msg="Event \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.021704666Z" level=warning msg="Event \"$gKNQBY9YipcsZKL-ZUQb_8KMeETcycGRkRsvBrBJD6g\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$gKNQBY9YipcsZKL-ZUQb_8KMeETcycGRkRsvBrBJD6g\" is not allowed by its auth_events: eventauth: \"@nex:dendrite.nexy7574.co.uk\" is not allowed to change their membership from \"leave\" to \"join\" as join rule \"invite\" forbids it"
time="2025-04-05T01:37:10.022904500Z" level=warning msg="Event \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\" is not allowed by its auth events" error="gomatrixserverlib: event with ID \"$ArLVQYeEb4YJzCmygZFBBj1Xaf6k1H6yz8nwb85EWBg\" is not allowed by its auth_events: illegal base64 data at input byte 0"
time="2025-04-05T01:37:10.023200155Z" level=warning msg="Discarding 5 auth/state event(s) due to invalid signatures" context=missing
time="2025-04-05T01:37:10.040428393Z" level=warning msg="Failed to join room through server" error="respSendJoin.Check: gomatrixserverlib: event with ID \"$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk\" is not allowed by the current room state: eventauth: \"@nex:dendrite.nexy7574.co.uk\" is not allowed to change their membership from \"leave\" to \"join\" as join rule \"invite\" forbids it" room_id="!P9ijcKs2PjqQUtoEEY:nexy7574.co.uk" server_name=nexy7574.co.uk
time="2025-04-05T01:37:10.040494017Z" level=error msg="failed to join user \"@nex:dendrite.nexy7574.co.uk\" to room \"!P9ijcKs2PjqQUtoEEY:nexy7574.co.uk\" through 1 server(s): last error respSendJoin.Check: gomatrixserverlib: event with ID \"$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk\" is not allowed by the current room state: eventauth: \"@nex:dendrite.nexy7574.co.uk\" is not allowed to change their membership from \"leave\" to \"join\" as join rule \"invite\" forbids it"
time="2025-04-05T01:37:10.040523088Z" level=error msg="Failed to join room" error="contents=[] msg=respSendJoin.Check: gomatrixserverlib: event with ID \"$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk\" is not allowed by the current room state: eventauth: \"@nex:dendrite.nexy7574.co.uk\" is not allowed to change their membership from \"leave\" to \"join\" as join rule \"invite\" forbids it code=0 wrapped=" room_id="#the-fish-room:nexy7574.co.uk" servers="[]" user_id="@nex:dendrite.nexy7574.co.uk"
time="2025-04-05T01:37:10.040578589Z" level=error msg="Failed to marshal JSONResponse" error="json: error calling MarshalJSON for type json.RawMessage: invalid character 'r' looking for beginning of value" req.id=ptl6RXnApOEM req.method=POST req.path="/_matrix/client/v3/join/#the-fish-room:nexy7574.co.uk"
time="2025-04-05T01:37:10.052808425Z" level=warning msg="Auth event $mZwQzOz0RbOGs-2ZwVQ9wv4zCaGHkAAg7bwsn1L94oE rejected" context=missing error="illegal base64 data at input byte 0" event_id="$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk" kind=KindNew origin=nexy7574.co.uk room_id="!P9ijcKs2PjqQUtoEEY:nexy7574.co.uk" type=m.room.member
time="2025-04-05T01:37:10.135727624Z" level=warning msg="Auth event $pAn0iqAyn87gmnU7QSkxndcPEvAOz48gI8IuL3wRV6w rejected" context=missing error="illegal base64 data at input byte 0" event_id="$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk" kind=KindNew origin=nexy7574.co.uk room_id="!P9ijcKs2PjqQUtoEEY:nexy7574.co.uk" type=m.room.member
time="2025-04-05T01:37:10.139510087Z" level=warning msg="Roomserver failed to process event" error="r.fetchAuthEvents: auth event ID $DuzOclVvmyTX5HwDJPSGU6yxT3HYrmsMZw9vE5HzzlE not known but should be" event_id="$93-w6ebaYbrK9ahoy4sK2eW1TiIYQqTXTL27MGQcXZk" room_id="!P9ijcKs2PjqQUtoEEY:nexy7574.co.uk" type=m.room.member

The state for this room is also attached. I'm aware that this is just a case of "problem exists between chair and computer", but this does carry the capacity to accidentally brick a room in the eyes of Dendrite servers accidentally (the room I bricked with this was also entirely accidentally, I knew gmsl didn't handle UTF-8 very well but it never crossed my mind that it'd affect the auth chain)

state-pretty.json