Description
Is your feature request related to a problem? Please describe.
While adding some integration tests for our Akka.Cluster.Sharding persistence cleanup tool, I ran into a small problem on this PR: petabridge/Akka.Cluster.Sharding.RepairTool#7 - Akka.Persistence.Redis' IReadJournal does not support ICurrentPersistenceIds().
I know we removed it as part of #126 because we thought it'd be too expensive to implement in clustered redis scenarios, but I wonder if that's true.
Describe the solution you'd like
The "all events" and "events by tag" queries are out of the question - they're expensive because they can't effectively be cleaned up in a clustered environment without some kind of inter-node index and we're not database architects so we're not going to build one. Users can use a relational database for that.
But Akka.Persistence requires us to keep track of all used PersistentIds even after the entities have had all of their data purged - we're required to keep those records in perpetuity and therefore cleanup is not an issue.
So, I suppose we can try implementing the ICurrentPersistentIds and IPersistentIds queries in one of two ways:
- Scatter-gather to all of the nodes in the cluster each time we need to run one of those queries and just use the built-in key indicies we have now. That's O(n) where n = entity count.
- Create a special table that only really needs to get replicated to single entity and have it contain a hashset that has all of the entity ids and we can use a Redis channel to receive notifications when that entity is updated. I don't really like this idea because it's single point of failure.
Describe alternatives you've considered
We could also go on not doing this, but since we're taking a dependency on ICurrentPersistentIds for helping Cluster.Sharding users cleanup I'd like to do our best to support it.