[exporter/elasticsearch] Change default mapping mode to `otel`

[carsonip]

Component(s)

exporter/elasticsearch

Is your feature request related to a problem? Please describe.

None and raw mapping mode are not useful in kibana UI

Describe the solution you'd like

change mapping mode default to otel to deliver better user experience out of the box, without deprecating existing mapping modes

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/elasticsearch: @JaredTan95 @carsonip @lahsivjar

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[carsonip]

/label -needs-triage

[felixbarny]

This may be a bit of a controversial issue. I'd specifically like to get input from @JaredTan95 of your usage of the none and raw mapping modes. Would it be feasible for the use cases you know about to switch to the otel mapping mode?

[carsonip]

pinging @JaredTan95 again in case the previous notification was missed

[JaredTan95]

raw mode was introduced by #29619, I think you can switch the default mode to otel( better user experience in kibana case), but other modes should not be deprecated.

[felixbarny]

@ycombinator do you think the raw mode is still needed or is it superseded by bodymap and ecs?

[carsonip]

I've updated this issue to not require a deprecation of any mapping modes (let's do it at any other time), but to just change the default mapping mode to otel. I wonder if #37283 is a blocker.

[felixbarny]

The simplest would be to just document ES 8.12 to be a requirement for the default config. In order to get proper mappings in otel mode, you need to use ES 8.16+ anyway. Unless we're removing mapping modes, users can still choose another mapping mode that doesn't use require_data_stream. We need to mark this change as a breaking change either way. Users that wish to keep the none mapping mode will need to set this explicitly.

If it turns out to be important to support older versions of ES with the otel mapping mode, we could also just add a config option to disable require_data_stream. At least, that will make it possible to use with older ES versions. At that point, we'll probably also want to document the mappings you should apply. But at this point, I'm not convinced that this is urgent.

Bottom line is that I don't think it's a blocker.

[carsonip]

sgtm, let's proceed with that.