Anything with /datum/component/swarming fails to GC

[GDNgit]

Issue Description:
Anything with /datum/component/swarming fails to GC, this includes Bees, Spiderlings, Viscerators, and Hivelords.

What did you expect to happen:
The above properly GC

What happened instead:
GC failure

Why is this bad/What are the consequences:
things not GCing bad

Steps to reproduce the problem:
Spawn something with /datum/component/swarming
Destroy it
Check simple Del log after awhile

When did the problem start happening:
This specific issue happened due to #13583 , however it's actually due to a larger bug with component registration itself
(All of this below was found by @warriorstar-orion, not me all credit goes to them, this basically mirrors what they said on discord)
Multiple /datum/component can be added to an ATOM, and are stored using a key value, however this key value also checks for /datum/component itself, leading to /datum/component being added to the list. This causes a reference to not properly be removed, and the atom to not GC. (Hopefully this properly reflects what the issue is)

[farie82]

Last 15 days it failed almost all of the time they get qdeleted.

When enabling tracking (even as QDEL hint) it finds no references:

So this probably means that the cause of the failure is in the proc that kills it, which is most likely process. And by god is this an ugly proc which does too much and uses sleep and spawn often. There is also a poll being started. Yet this ain't the cause of the failure.