Description
I find that the example code, at least running on my cloud, is unable to keep up with the firehose. There is not a network issue -- 30 Mbit/s is not enough for me to even notice (I run the network). I see that the node process has pegged the CPU, and the generator's subscription processing is running about 24h behind real time. The generator is able to process about 450 events per second by my measurements (most of which, of course, will be ignored -- I already modified the code to reduce the amount of garbage generated for like and follow events, which the generator doesn't examine). It would seem that using the recently introduced Jetstream would greatly reduce the amount of wasted processing performed in the underlying subscription processing, and might allow my poor cloud VM to actually perform the real-time service it was intended to. I suspect that many people who are starting from this example code are in a similar situation, banging their head against the wall because it doesn't appear to work, when in fact it's just getting more and more backlogged.