Unpleasent behavior when enrolling to non existing policy

[michalpristas]

When agent we try to add fleet-server agent specifying -fleet-server-policy with an incorrect value (e.g typo) agent misbehaves.

It spins up fleet server, fleet server tries to read policy index with a specified policy which is empty, and retries and retries

behavior is not well defined:

m.reporter.UpdateState(client.UnitStateStarting, fmt.Sprintf("Waiting on policy with Fleet Server integration: %s", m.policyID), nil) //nolint:errcheck // not clear what to do in failure cases

there may be a memory leak involved

[michel-laterman]

This is expected behaviour from fleet-server.
The (policy) self monitor is used by fleet-server to ensure that the policy index exists, and the specified policy can be found with a fleet-server integration in the index (https://github.com/elastic/fleet-server/blob/main/internal/pkg/policy/self.go#L70)

This is used during the bootstrapping process, the fleet-server will be starting indefinitely if it can't find it's policy.

The assumptions behind this were that fleet-server and kibana could startup concurrently in a cloud deployment and that fleet-server should retry in case the monitor is started before kibana finishes creating the policies index and writing the initial policy.

[jlind23]

Shall we then close as won't fix?