[cmd/opampsupervisor] Remote Configuration model in the supervisor

[canthar]

Component(s)

cmd/opampsupervisor

Describe the issue you're reporting

Hi!
I'm writing here to start a discussion about how opampsupervisor implements RemoteConfig OpAMP feature.

Just from OpAMP point of view, remote configuration is a map with filenames as keys. It makes sense, as it makes it possible to reflect this configuration on the filesystem. When using with OTel Collectors, it'd be possible to include other files (so to use exporters: ${file:exporters.yaml} syntax).

However, I see it was decided that opampsupervisor implements quite a different model. It takes all the configuration files from the RemoteConfig, sorts them by filename and merges all the files into a final one (with the possibility to also modify the final configuration locally).

This supervisor's model works fine as it makes it possible to configure the collector. However, my concern is that this behavior changes the meaning of the OpAMP message, as now the final configuration depends on how the files are sorted. In other words, it's not behaving like a map as defined by the protocol, but a list.

This change is important for any OpAMP server implementation, so that they could present the remote configuration to the user in the same way as it’s interpreted.

That’s why I have a question, what is the reason why this change of meaning was implemented here, especially that opampsupervisor is in fact the reference implementation of the supervisor model in OpAMP. Do you think, that this model should be the standard one for configuring OTel Collectors?
As OpenTelemetry is the main driver of OpAMP, maybe this behavior should be mentioned or somehow explicitly stated in the protocol?

[github-actions]

Pinging code owners:

• cmd/opampsupervisor: @evan-bradley @atoulme @tigrannajaryan

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[douglascamata]

@canthar: you have a great point here. I think the current logic is a consequence of not saving any of the files in the incoming configmap to disk. For now, they are read into memory only so doing something like exporters: ${file:exporters.yaml} is not possible.

When we start to persist the configurations to disk and delegate the merging to the Collector itself this will become possible, but it raises an important question: how to know which files from the configmap should be only saved to disk and which ones should loaded as configuration by the Collector?

So in the end I think instead of doing exporters: ${file:exporters.yaml} the OpAMP backend should just send the configuration of exporters in a separate file, which will then be loaded by the Collector via another --config entry.

[douglascamata]

For more history on the configmap merging, see #34608 and open-telemetry/opentelemetry-collector#8754