Skip to content

Commit e668f3a

Browse files
edmocostaf7o
authored andcommitted
[pkg/ottl] Add support for localized time parsing into the timeutils (open-telemetry#34353)
**Description:** <Describe what has changed.> Added support for localized time parsing into the `coreinternal/timeutils` package. The [tracking issue](open-telemetry#32977) is a following up to open-telemetry#32140, and the added function (`ParseLocalizedStrptime`) can be used later to [add locale support](open-telemetry#32978) to the ottl `Time` function. As discussed in the related issues, the plan is to have a similar support as implemented by the library [monday](https://github.com/goodsign/monday), making it possible to parse non-english time strings into `time.Time` objects. Elastic has built a new OSS library for that same purpose ([lunes](https://github.com/elastic/lunes)), that considering the discussed requirements, seems to be a better option than `monday`. Both libraries are just wrapper to the `time.Parse` and `time.ParseInLocation` features. They work by translating the foreign language value to English before calling the standard parsing functions. The main `lunes` differences are: 1 - Performance is considerably better, ~13x faster for complete .Parse operations: ``` BenchmarkParseLunes-10 2707008 429.7 ns/op 220 B/op 5 allocs/op BenchmarkParseMonday-10 212086 5630 ns/op 3753 B/op 117 allocs/op BenchmarkParseInLocationLunes-10 2777323 429.4 ns/op 220 B/op 5 allocs/op BenchmarkParseInLocationMonday-10 210357 5596 ns/op 3754 B/op 117 allocs/op ``` Given `ParseLocalizedStrptime` uses `lunes.Translate` under the hood, its performance is similar to the existing `ParseStrptime`, adding an extra overhead of ~303 ns/op for translating the value before parsing: ``` BenchmarkTranslate-10 3591234 302.4 ns/op 220 B/op 5 allocs/op ``` ``` BenchmarkParseLocalizedStrptime-10 821572 1405 ns/op 429 B/op 9 allocs/op BenchmarkParseStrptime-10 1000000 1082 ns/op 186 B/op 6 allocs/op ``` 2 - Translations are based on the [CLDR](https://cldr.unicode.org/) project, and it does support 900+ locales (vs 45+), including locales in draft stage. Those lunes translations are [generated](https://github.com/elastic/lunes/blob/main/generator.go) from a CLDR core file, and does not require manually changes. 3 - Replicates all the relevant standard `time.format_test.go` test cases, parsing foreign language values with and without layout replacements in all available locales and supported layouts (https://github.com/elastic/lunes/blob/main/lunes_test.go#L154). 4 - It is actively maintained (hosted under elastic repo). **Link to tracking Issue:** open-telemetry#32977 **Testing:** - Added unit tests For now, the only way of manually testing this functionality is by invoking the `ParseLocalizedStrptime` function manually through tests. **Documentation:**
1 parent c00c4db commit e668f3a

File tree

113 files changed

+418
-143
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

113 files changed

+418
-143
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: pkg/ottl
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Add support for localized time parsing into the coreinternal/timeutils
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues:
14+
- 32977
15+
16+
# (Optional) One or more lines of additional information to render under the primary note.
17+
# These lines will be padded with 2 spaces and then inserted directly into the document.
18+
# Use pipe (|) for multiline entries.
19+
subtext:
20+
21+
# If your change doesn't affect end users or the exported elements of any package,
22+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
23+
# Optional: The change log or logs in which this entry should be included.
24+
# e.g. '[user]' or '[user, api]'
25+
# Include 'user' if the change is relevant to end users.
26+
# Include 'api' if there is a change to a library API.
27+
# Default: '[user]'
28+
change_logs: [api]

cmd/otelcontribcol/go.mod

+2-1
Original file line numberDiff line numberDiff line change
@@ -439,7 +439,7 @@ require (
439439
github.com/containerd/ttrpc v1.2.4 // indirect
440440
github.com/coreos/go-oidc/v3 v3.11.0 // indirect
441441
github.com/coreos/go-systemd/v22 v22.5.0 // indirect
442-
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
442+
github.com/cyphar/filepath-securejoin v0.2.5 // indirect
443443
github.com/danieljoos/wincred v1.1.2 // indirect
444444
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
445445
github.com/dennwc/varint v1.0.0 // indirect
@@ -464,6 +464,7 @@ require (
464464
github.com/elastic/go-structform v0.0.12 // indirect
465465
github.com/elastic/go-sysinfo v1.7.1 // indirect
466466
github.com/elastic/go-windows v1.0.1 // indirect
467+
github.com/elastic/lunes v0.1.0 // indirect
467468
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
468469
github.com/envoyproxy/go-control-plane v0.12.1-0.20240621013728-1eb8caab5155 // indirect
469470
github.com/envoyproxy/protoc-gen-validate v1.0.4 // indirect

cmd/otelcontribcol/go.sum

+4-2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

cmd/oteltestbedcol/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ require (
8989
github.com/docker/go-units v0.5.0 // indirect
9090
github.com/elastic/go-grok v0.3.1 // indirect
9191
github.com/elastic/go-structform v0.0.12 // indirect
92+
github.com/elastic/lunes v0.1.0 // indirect
9293
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
9394
github.com/envoyproxy/go-control-plane v0.12.1-0.20240621013728-1eb8caab5155 // indirect
9495
github.com/envoyproxy/protoc-gen-validate v1.0.4 // indirect

cmd/oteltestbedcol/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

connector/countconnector/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ require (
2525
github.com/cespare/xxhash/v2 v2.3.0 // indirect
2626
github.com/davecgh/go-spew v1.1.1 // indirect
2727
github.com/elastic/go-grok v0.3.1 // indirect
28+
github.com/elastic/lunes v0.1.0 // indirect
2829
github.com/go-logr/logr v1.4.2 // indirect
2930
github.com/go-logr/stdr v1.2.2 // indirect
3031
github.com/go-viper/mapstructure/v2 v2.1.0 // indirect

connector/countconnector/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

connector/datadogconnector/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ require (
119119
github.com/docker/go-units v0.5.0 // indirect
120120
github.com/dustin/go-humanize v1.0.1 // indirect
121121
github.com/elastic/go-grok v0.3.1 // indirect
122+
github.com/elastic/lunes v0.1.0 // indirect
122123
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
123124
github.com/fatih/color v1.16.0 // indirect
124125
github.com/felixge/httpsnoop v1.0.4 // indirect

connector/datadogconnector/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

connector/routingconnector/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ require (
2121
github.com/cespare/xxhash/v2 v2.3.0 // indirect
2222
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
2323
github.com/elastic/go-grok v0.3.1 // indirect
24+
github.com/elastic/lunes v0.1.0 // indirect
2425
github.com/go-logr/logr v1.4.2 // indirect
2526
github.com/go-logr/stdr v1.2.2 // indirect
2627
github.com/go-viper/mapstructure/v2 v2.1.0 // indirect

connector/routingconnector/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

connector/sumconnector/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ require (
2222
github.com/cespare/xxhash/v2 v2.3.0 // indirect
2323
github.com/davecgh/go-spew v1.1.1 // indirect
2424
github.com/elastic/go-grok v0.3.1 // indirect
25+
github.com/elastic/lunes v0.1.0 // indirect
2526
github.com/go-logr/logr v1.4.2 // indirect
2627
github.com/go-logr/stdr v1.2.2 // indirect
2728
github.com/go-viper/mapstructure/v2 v2.1.0 // indirect

connector/sumconnector/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/datadogexporter/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,7 @@ require (
163163
github.com/docker/go-units v0.5.0 // indirect
164164
github.com/dustin/go-humanize v1.0.1 // indirect
165165
github.com/elastic/go-grok v0.3.1 // indirect
166+
github.com/elastic/lunes v0.1.0 // indirect
166167
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
167168
github.com/envoyproxy/go-control-plane v0.12.1-0.20240621013728-1eb8caab5155 // indirect
168169
github.com/envoyproxy/protoc-gen-validate v1.0.4 // indirect

exporter/datadogexporter/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/datadogexporter/integrationtest/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@ require (
140140
github.com/docker/go-units v0.5.0 // indirect
141141
github.com/dustin/go-humanize v1.0.1 // indirect
142142
github.com/elastic/go-grok v0.3.1 // indirect
143+
github.com/elastic/lunes v0.1.0 // indirect
143144
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
144145
github.com/envoyproxy/go-control-plane v0.12.1-0.20240621013728-1eb8caab5155 // indirect
145146
github.com/envoyproxy/protoc-gen-validate v1.0.4 // indirect

exporter/datadogexporter/integrationtest/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/elasticsearchexporter/integrationtest/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ require (
4646
github.com/elastic/go-structform v0.0.12 // indirect
4747
github.com/elastic/go-sysinfo v1.14.0 // indirect
4848
github.com/elastic/go-windows v1.0.1 // indirect
49+
github.com/elastic/lunes v0.1.0 // indirect
4950
github.com/expr-lang/expr v1.16.9 // indirect
5051
github.com/felixge/httpsnoop v1.0.4 // indirect
5152
github.com/fsnotify/fsnotify v1.7.0 // indirect

exporter/elasticsearchexporter/integrationtest/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/honeycombmarkerexporter/go.mod

+1
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ require (
2727
github.com/cespare/xxhash/v2 v2.3.0 // indirect
2828
github.com/davecgh/go-spew v1.1.1 // indirect
2929
github.com/elastic/go-grok v0.3.1 // indirect
30+
github.com/elastic/lunes v0.1.0 // indirect
3031
github.com/felixge/httpsnoop v1.0.4 // indirect
3132
github.com/fsnotify/fsnotify v1.7.0 // indirect
3233
github.com/go-logr/logr v1.4.2 // indirect

exporter/honeycombmarkerexporter/go.sum

+2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/rabbitmqexporter/go.mod

+2-2
Original file line numberDiff line numberDiff line change
@@ -97,12 +97,12 @@ require (
9797
go.opentelemetry.io/otel/trace v1.29.0 // indirect
9898
go.uber.org/multierr v1.11.0 // indirect
9999
golang.org/x/crypto v0.26.0 // indirect
100-
golang.org/x/mod v0.17.0 // indirect
100+
golang.org/x/mod v0.19.0 // indirect
101101
golang.org/x/net v0.28.0 // indirect
102102
golang.org/x/sync v0.8.0 // indirect
103103
golang.org/x/sys v0.24.0 // indirect
104104
golang.org/x/text v0.17.0 // indirect
105-
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
105+
golang.org/x/tools v0.23.0 // indirect
106106
google.golang.org/genproto/googleapis/rpc v0.0.0-20240822170219-fc7c04adadcd // indirect
107107
google.golang.org/grpc v1.66.0 // indirect
108108
google.golang.org/protobuf v1.34.2 // indirect

exporter/rabbitmqexporter/go.sum

+4-4
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

exporter/signalfxexporter/go.mod

+2
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ require (
3838
github.com/beorn7/perks v1.0.1 // indirect
3939
github.com/cespare/xxhash/v2 v2.3.0 // indirect
4040
github.com/davecgh/go-spew v1.1.1 // indirect
41+
github.com/elastic/lunes v0.1.0 // indirect
4142
github.com/felixge/httpsnoop v1.0.4 // indirect
4243
github.com/fsnotify/fsnotify v1.7.0 // indirect
4344
github.com/go-logr/logr v1.4.2 // indirect
@@ -54,6 +55,7 @@ require (
5455
github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
5556
github.com/knadh/koanf/v2 v2.1.1 // indirect
5657
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
58+
github.com/magefile/mage v1.15.0 // indirect
5759
github.com/mitchellh/copystructure v1.2.0 // indirect
5860
github.com/mitchellh/reflectwalk v1.0.2 // indirect
5961
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect

0 commit comments

Comments
 (0)