Retention policy purge jobs fail if `run_background_tasks_on` is set to something other than master

[babolivier]

If run_background_tasks_on is set to anything other than master, Synapse will try to run background jobs on a worker, including the message retention policies support's purge jobs. However, these jobs end up calling the purge_history function of the storage layer (in the PurgeEventsStorage class) which doesn't exist on a worker, therefore the jobs fail with AttributeError: 'GenericWorkerSlavedStore' object has no attribute 'purge_history'.

This bug was introduced in #8513

cc @iansinclair

[clokep]

I guess we either need to move the method to the worker store or backout the changes that run that on a background worker. I think they're potentially heavy though, so if we could run it on a worker that would be best!

[babolivier]

Yes I tend to agree - though having something that fiddles with the database as much as purge_history does run on a worker sounds a bit scary to me.

[callahad]

Noting that @richvdh self-assigned: I believe you're a bit overloaded at the moment; would it make sense to hand this off to someone else (@DMRobertson?) with you providing review once impl'd?

[richvdh]

Noting that @richvdh self-assigned:

sorry, at one point it sounded like this was a drop-the-world level of urgency, but that turned out to be incorrect. Yes, it's unlikely I'll have time to look at this in the near future!

[callahad]

Let's tentatively try to address this after FOSDEM next year... noting that retention policies are still an experimental feature themselves.