Sometimes no response from other servers to fetchSockets or serverSideEmit #32
Description
My team has a multi-server deployment, and we use the Redis Streams Adapter with Valkey (but have previously used and seen this issue on Redis as well) for our sockets communication across those servers.
We occasionally get reports from users that they are not seeing the messages of other users. When looking into one of such reports, I noticed that the two members in the chat were each connected successfully to sockets, but to two different servers, which would indicate a potential problem with the streams adapter communication between the two servers.
I am able to reliably reproduce the issue by restarting the Valkey instance - i.e. restart the Valkey/Redis instance, observe that any fetchSockets or serverSideEmit does not receive any responses. The issue, however, definitely occurs more often than just a restart of the Valkey/Redis instance, I just unfortunately have not figured out exactly the conditions during which it happens.
The most odd of it all, though, is that though the socket servers won't respond, they are still sending a heartbeat into the stream.
In the image below, I had set up a separate process with an interval that would simply send a ping (via serverSideEmit) into the stream for my socket servers to respond to. The servers were at first not responding, so I restarted them, thus "resetting" the streams adapter connections.
