Random state parameter for non-deterministic equivalence oracles

[steve-anunknown]

I was thinking that it would be appropriate for non-deterministic equivalence oracles, like those that perform random walks for example, to accept a 'random state' or 'seed' parameter and initialize their own random number generator using this parameter. That way, users of the library can have reproducible results even when running multiple learning experiments in parallel.

For example, the StatePrefixEqOracle would be modified like this:

class StatePrefixEqOracle(Oracle):
    def __init__(self, alphabet: list, sul: SUL, walks_per_round, walks_per_state=10, walk_len=12, depth_first=False, 
						seed=42):
        """
        Args:

            alphabet: input alphabet

            sul: system under learning

            walks_per_state:individual walks per state of the automaton over the whole learning process

            walk_len:length of random walk

            depth_first:first explore newest states
        """

        super().__init__(alphabet, sul)
        self.walks_per_state = walks_per_state
        self.walks_per_round = walks_per_round
        self.steps_per_walk = walk_len
        self.depth_first = depth_first
        self.freq_dict = dict()
		self.rng = random.Random(seed) # <- added this line

and sampling would be done like this:

            for _ in range(self.steps_per_walk):
                suffix += (self.rng.choice(self.alphabet),)
				# instead of suffix += (random.choice(self.alphabet),)

[emuskardin]

Hi,

To be honest, I think this adds complexity that is better kept outside of the oracle.

Maybe fix the seed once again just before the learning algorithm starts or something like that.

What is the differance to doing seed(1) before algorithm invocation?

[steve-anunknown]

I think that there exists a difference only if you consider running learning experiments in parallel.

I came up with this idea because I was benchmarking some oracles and I wanted to have reproducible results. Running them in parallel was crucial because otherwise the duration of the experiments would be too much.

I think that, due to non deterministic scheduling of threads, globally setting the seed will not work. That's why I introduced the seed parameter.

My concern was that, if a user of the library wanted to run learning experiments in parallel and have reproducible results, they would not be able to do so without modifying the source code. Is there any other way to achieve what I'm saying?

Of course, if parallelism is not considered, I agree with your point that simply setting the seed is simpler.

[emuskardin]

I see.

I am not for including it in oracles, as the current seed stuff works well, and adds (rarely necessary) complexity to almost all oracles.

Currently, parallelism is not one of the things I am too interested it, maybe in some feature releases.
As I am thinking form the perspective of the most common use-case, and there parallelism is not needed and would make it quite more end-user intensive unless properly designed.