Akka.Cluster.Tools: fix mutability and oldest state bugs with ClusterSingletonManager (#7298)

* fix mutability bugs with `ClusterSingletonManager`

Looks like the list of available nodes was getting mangled - seems like a basic mutability issue inside the `ClusterSingletonManager`.

* fixed another mutability error

* explicitly re-order "oldest nodes" based on data received from OldestChangedBuffer

* WIP nullability

* fixed nullability issues and API approvals

* removed debug logging

* removed old code

* fixed some nullability issues

* fix racy cluster singleton restart spec

Author: Aaronontheweb

src/contrib/cluster/Akka.Cluster.Tools.Tests/Singleton/ClusterSingletonRestart2Spec.cs

@@ -31,7 +31,7 @@ public ClusterSingletonRestart2Spec(ITestOutputHelper output) : base("""
                                                                                            akka.loglevel = INFO
                                                                                            akka.actor.provider = "cluster"
                                                                                            akka.cluster.roles = [singleton]
-                                                                                           #akka.cluster.auto-down-unreachable-after = 2s
+                                                                                           akka.cluster.split-brain-resolver.stable-after = 2s
                                                                                            akka.cluster.singleton.min-number-of-hand-over-retries = 5
                                                                                            akka.remote {
                                                                                              dot-netty.tcp {
@@ -120,7 +120,7 @@ await WithinAsync(TimeSpan.FromSeconds(5), () =>
             await Join(_sys4, _sys3);
 
             // let it stabilize
-            //Task.Delay(TimeSpan.FromSeconds(5)).Wait();
+            await Task.Delay(TimeSpan.FromSeconds(5));
 
             await WithinAsync(TimeSpan.FromSeconds(10), () =>
             {