[pkg/ottl] Add OTTL support for profiles #37574

rockdaboot · 2025-01-29T17:44:51Z

This is a first PR to get profiles support into OTTL. Follow-up PRs will add support for more sub-types of profiles like sample, mapping etc..

dmathieu

This is a lot of very similar code. But maybe that's how OTTL does things.

TylerHelmuth · 2025-02-19T21:12:04Z

pkg/ottl/contexts/ottlprofile/profile.go

+	_ internal.InstrumentationScopeContext = (*TransformContext)(nil)
+	_ internal.ProfileContext              = TransformContext{}
+
+	// _ zapcore.ObjectMarshaler              = (*TransformContext)(nil)


The TransformContext should implement the zapcore.ObjectMarshaler so that debug logging works.

Q: Would you like to see a meaningful marshalling, for example seeing the actual attributes for a sample? Or instead, see the attribute indices for a sample.
I would prefer the first, otherwise it takes a lot of mental load to translate the indices into attributes with the help of the attribute table. It is normally a human who does the debugging.

The original intent for this printout feature was to show the data exactly how OTTL sees it. This way when debugging a statement a user can see what the collector saw instead of what their mental model is of the data.

So I think it should printout indices and attributes separately, but if that is completely worthless to a user then we could do some manipulation. Can you provide a rough example of what the printout would look like for the 2 options you proposed?

pkg/ottl/contexts/ottlprofile/profile.go

pkg/ottl/contexts/internal/profile.go

hfakar · 2025-04-09T11:39:04Z

Is there any updates on this?

rockdaboot · 2025-04-10T14:19:36Z

Is there any updates on this?

@hfakar This PR is waiting for #38928 to be merged 🤷🏻

…ssors (#38928)  #### Description Currently, OTTL contexts setters can received different data type when using slices, which might lead to unexpected outcomes, for example, slice setters functions can receive the following value types: - `[]any` when set from literals: `set(string_table, ["foo", "bar"])` - integer values within the `[]any` are parsed as `int64` - `[]T` when copied from another path: `set(slice_path, slice_path)` - `pcommon.Slice` when set from an map's key value: `set(slice_path, resource.attributes["slice"])` Those different possibilities must be properly handled on the setter assessors, and are not obvious since the value is any-wrapped. Failing to handle all those possible types results in the path not being set or errors. This PR adds a set of functions that helps setting and getting slices values, making it easier handling them until OTTL slice parsing is not standardized. Those functions will be especially useful for the new profile context (#37574), which needs to provide access to different pdata typed slices. We've recently done something similar to maps (#38434). Usage examples: ```go // tCtx.GetProfile().StringTable() -> pcommon.StringSlice func accessStringTableKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys, val) }, } } func accessStringTable[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return tCtx.GetProfile().StringTable().AsRaw(), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValues[string](tCtx.GetProfile().StringTable(), val) }, } } // *Int* function versions also handle type conversions and ensure returned values are compatible with OTTL defaults (int64). // tCtx.GetProfile().LocationIndices() -> pcommon.Int32Slice func accessLocationIndicesKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys, val) }, } } func accessLocationIndices[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices()), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices(), val) }, } } ```  #### Testing - Unit tests  --------- Co-authored-by: Tim Rühsen <[email protected]>

atoulme · 2025-04-11T18:56:37Z

No longer waiting for a PR to be merged, but conflicts need to be resolved. Please attend to those and mark the PR ready as review again.

atoulme · 2025-04-11T18:57:12Z

Removing waiting-for-code-owners as the PR was approved by @TylerHelmuth

…ssors (open-telemetry#38928)  #### Description Currently, OTTL contexts setters can received different data type when using slices, which might lead to unexpected outcomes, for example, slice setters functions can receive the following value types: - `[]any` when set from literals: `set(string_table, ["foo", "bar"])` - integer values within the `[]any` are parsed as `int64` - `[]T` when copied from another path: `set(slice_path, slice_path)` - `pcommon.Slice` when set from an map's key value: `set(slice_path, resource.attributes["slice"])` Those different possibilities must be properly handled on the setter assessors, and are not obvious since the value is any-wrapped. Failing to handle all those possible types results in the path not being set or errors. This PR adds a set of functions that helps setting and getting slices values, making it easier handling them until OTTL slice parsing is not standardized. Those functions will be especially useful for the new profile context (open-telemetry#37574), which needs to provide access to different pdata typed slices. We've recently done something similar to maps (open-telemetry#38434). Usage examples: ```go // tCtx.GetProfile().StringTable() -> pcommon.StringSlice func accessStringTableKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys, val) }, } } func accessStringTable[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return tCtx.GetProfile().StringTable().AsRaw(), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValues[string](tCtx.GetProfile().StringTable(), val) }, } } // *Int* function versions also handle type conversions and ensure returned values are compatible with OTTL defaults (int64). // tCtx.GetProfile().LocationIndices() -> pcommon.Int32Slice func accessLocationIndicesKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys, val) }, } } func accessLocationIndices[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices()), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices(), val) }, } } ```  #### Testing - Unit tests  --------- Co-authored-by: Tim Rühsen <[email protected]>

Fixes open-telemetry#36104 This is a first PR to get profiles support into OTTL. Follow-up PRs will add support for more sub-types of profiles like `sample`, `mapping` etc..

…ssors (open-telemetry#38928)  #### Description Currently, OTTL contexts setters can received different data type when using slices, which might lead to unexpected outcomes, for example, slice setters functions can receive the following value types: - `[]any` when set from literals: `set(string_table, ["foo", "bar"])` - integer values within the `[]any` are parsed as `int64` - `[]T` when copied from another path: `set(slice_path, slice_path)` - `pcommon.Slice` when set from an map's key value: `set(slice_path, resource.attributes["slice"])` Those different possibilities must be properly handled on the setter assessors, and are not obvious since the value is any-wrapped. Failing to handle all those possible types results in the path not being set or errors. This PR adds a set of functions that helps setting and getting slices values, making it easier handling them until OTTL slice parsing is not standardized. Those functions will be especially useful for the new profile context (open-telemetry#37574), which needs to provide access to different pdata typed slices. We've recently done something similar to maps (open-telemetry#38434). Usage examples: ```go // tCtx.GetProfile().StringTable() -> pcommon.StringSlice func accessStringTableKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValue[K, string](ctx, tCtx, tCtx.GetProfile().StringTable(), keys, val) }, } } func accessStringTable[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return tCtx.GetProfile().StringTable().AsRaw(), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonTypedSliceValues[string](tCtx.GetProfile().StringTable(), val) }, } } // *Int* function versions also handle type conversions and ensure returned values are compatible with OTTL defaults (int64). // tCtx.GetProfile().LocationIndices() -> pcommon.Int32Slice func accessLocationIndicesKey[K ProfileContext](ctx context.Context, keys []ottl.Key[K]) ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys) }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValue[K, int32](ctx, tCtx, tCtx.GetProfile().LocationIndices(), keys, val) }, } } func accessLocationIndices[K ProfileContext]() ottl.StandardGetSetter[K] { return ottl.StandardGetSetter[K]{ Getter: func(_ context.Context, tCtx K) (any, error) { return ctxutil.GetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices()), nil }, Setter: func(_ context.Context, tCtx K, val any) error { return ctxutil.SetCommonIntSliceValues[int32](tCtx.GetProfile().LocationIndices(), val) }, } } ```  #### Testing - Unit tests  --------- Co-authored-by: Tim Rühsen <[email protected]>

Fixes open-telemetry#36104 This is a first PR to get profiles support into OTTL. Follow-up PRs will add support for more sub-types of profiles like `sample`, `mapping` etc..

github-actions bot added the pkg/ottl label Jan 29, 2025

github-actions bot requested review from bogdandrutu, edmocosta, evan-bradley, kentquirk and TylerHelmuth January 29, 2025 17:45

rockdaboot force-pushed the ottl-profile branch 4 times, most recently from cd7d67e to 7a7b0dc Compare January 31, 2025 17:39

rockdaboot changed the title ~~Add minimal support of OTTL for profiles~~ [pkg/ottl] Add support of OTTL for profiles Jan 31, 2025

rockdaboot changed the title ~~[pkg/ottl] Add support of OTTL for profiles~~ [pkg/ottl] Add OTTL support for profiles Jan 31, 2025

rockdaboot force-pushed the ottl-profile branch 3 times, most recently from 594f736 to a629ef0 Compare February 4, 2025 10:28

dmathieu reviewed Feb 4, 2025

View reviewed changes

rockdaboot force-pushed the ottl-profile branch 3 times, most recently from cb4a7c6 to fa2f67c Compare February 10, 2025 09:56

rockdaboot mentioned this pull request Feb 10, 2025

[pkg/ottl] Add support for profiles #36104

Closed

rockdaboot force-pushed the ottl-profile branch from fa2f67c to 1cd3020 Compare February 10, 2025 10:17

rockdaboot mentioned this pull request Feb 12, 2025

Is it possible to add a process name filter? open-telemetry/opentelemetry-ebpf-profiler#351

Closed

rockdaboot force-pushed the ottl-profile branch from 1cd3020 to f5cb385 Compare February 17, 2025 10:48

Add OTTL support for profiles

7a05b41

rockdaboot force-pushed the ottl-profile branch from f5cb385 to 7a05b41 Compare February 18, 2025 09:36

rockdaboot marked this pull request as ready for review February 18, 2025 09:45

rockdaboot requested a review from a team as a code owner February 18, 2025 09:45

github-actions bot assigned djaglowski Feb 18, 2025

TylerHelmuth reviewed Feb 19, 2025

View reviewed changes

Drop parseEnum()

8e38cb0

atoulme added the waiting-for-code-owners label Mar 21, 2025

This was referenced Mar 23, 2025

Weekly Report: 2025-03-16 - 2025-03-23 LucaLanziani/opentelemetry-collector-contrib#16

Closed

Weekly Report: 2025-03-16 - 2025-03-23 LucaLanziani/opentelemetry-collector-contrib#17

Closed

edmocosta mentioned this pull request Mar 24, 2025

[chore][pkg/ottl] Add OTTL context utilities for handling slices accessors #38928

Merged

github-actions bot mentioned this pull request Mar 25, 2025

Weekly Report: 2025-03-18 - 2025-03-25 #38935

Closed

This was referenced Mar 25, 2025

[processor/attributes] Add support for profiles #38964

Closed

[processor/transform] Add support for profiles signal #39036

Closed

rockdaboot added 2 commits March 28, 2025 17:35

Change Option to ottl.Option[TransformContext

13da60f

Merge remote-tracking branch 'upstream/main' into ottl-profile

84c1de5

This was referenced Apr 1, 2025

Weekly Report: 2025-03-25 - 2025-04-01 #39070

Closed

Weekly Report: 2025-04-01 - 2025-04-08 #39228

Closed

atoulme marked this pull request as draft April 11, 2025 18:56

atoulme removed the waiting-for-code-owners label Apr 11, 2025

rockdaboot added 2 commits April 13, 2025 19:12

Merge remote-tracking branch 'upstream/main' into ottl-profile

391b10f

Use new slice functions

53b3cf8

rockdaboot marked this pull request as ready for review April 13, 2025 17:56

github-actions bot assigned MovieStoreGuy Apr 13, 2025

atoulme approved these changes Apr 14, 2025

View reviewed changes

atoulme merged commit d0ffb46 into open-telemetry:main Apr 14, 2025
174 checks passed

github-actions bot added this to the next release milestone Apr 14, 2025

rockdaboot mentioned this pull request Jun 4, 2025

[pkg/ottl] Add support for profiles sub-types #40489

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[pkg/ottl] Add OTTL support for profiles #37574

[pkg/ottl] Add OTTL support for profiles #37574

Uh oh!

rockdaboot commented Jan 29, 2025 •

edited

Loading

Uh oh!

dmathieu left a comment

Uh oh!

TylerHelmuth Feb 19, 2025

Uh oh!

rockdaboot Feb 25, 2025

Uh oh!

TylerHelmuth Feb 28, 2025

Uh oh!

Uh oh!

Uh oh!

hfakar commented Apr 9, 2025

Uh oh!

rockdaboot commented Apr 10, 2025

Uh oh!

atoulme commented Apr 11, 2025

Uh oh!

atoulme commented Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

[pkg/ottl] Add OTTL support for profiles #37574

[pkg/ottl] Add OTTL support for profiles #37574

Uh oh!

Conversation

rockdaboot commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dmathieu left a comment

Choose a reason for hiding this comment

Uh oh!

TylerHelmuth Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

rockdaboot Feb 25, 2025

Choose a reason for hiding this comment

Uh oh!

TylerHelmuth Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hfakar commented Apr 9, 2025

Uh oh!

rockdaboot commented Apr 10, 2025

Uh oh!

atoulme commented Apr 11, 2025

Uh oh!

atoulme commented Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

rockdaboot commented Jan 29, 2025 •

edited

Loading