Skip to content

feat: add splunkhec receiver #3440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
4a0ce64
Setup boilerplate for splunk hec otel receiver
kalleep Apr 23, 2025
40ff753
Add arguments
kalleep Apr 23, 2025
37c35cb
Extract replicated function
kalleep Apr 24, 2025
5056eed
Add tests
kalleep Apr 24, 2025
0f23c1b
Add converter
kalleep Apr 24, 2025
60e99d8
Add changelog
kalleep Apr 24, 2025
9ef6153
Fix table format
kalleep Apr 24, 2025
807703e
Add docs
kalleep Apr 24, 2025
a47edf0
Add reference
kalleep Apr 24, 2025
5c392b9
Add support for AccessTokenPassthrough
kalleep Apr 24, 2025
2b5691d
Fix changelog
kalleep Apr 24, 2025
d6e96a6
Restructure
kalleep Apr 24, 2025
29d5e15
Fix test
kalleep Apr 24, 2025
470c7c1
fix changelog
kalleep Apr 24, 2025
6cb015e
trigger ci
kalleep Apr 24, 2025
6507cb3
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
f5bc00a
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
2d3bebf
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
56cd689
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
3758ee1
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
bcb3e88
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
559f2bf
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
277bb95
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
4c417f5
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
9eff146
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
71a17a1
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
ab6e9e8
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
3280c54
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
13011d9
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
875155a
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
643862e
Update docs/sources/reference/components/otelcol/otelcol.receiver.inf…
kalleep Apr 25, 2025
2b62d68
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 25, 2025
ad4e042
Update docs/sources/reference/components/otelcol/otelcol.receiver.inf…
kalleep Apr 25, 2025
e688499
Fix flaky test
kalleep Apr 28, 2025
ebe273d
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 29, 2025
b4cf0c1
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 29, 2025
44b10b4
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 29, 2025
d7ff54f
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 29, 2025
8d3048c
Update docs/sources/reference/components/otelcol/otelcol.receiver.spl…
kalleep Apr 29, 2025
ffd81ed
Update splunkhec_test.go
kalleep May 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Main (unreleased)

- Add `validate` command to alloy that will perform limited validation of alloy configuration files. (@kalleep)

- Add `otelcol.receiver.splunkhec` component to receive events in splunk hec format and forward them to other `otelcol.*` components. (@kalleep)

### Enhancements

- Add binary version to constants exposed in configuration file syntatx. (@adlots)
Expand Down
1 change: 1 addition & 0 deletions docs/sources/reference/compatibility/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,7 @@ The following components, grouped by namespace, _consume_ OpenTelemetry `otelcol
- [otelcol.receiver.otlp](../components/otelcol/otelcol.receiver.otlp)
- [otelcol.receiver.prometheus](../components/otelcol/otelcol.receiver.prometheus)
- [otelcol.receiver.solace](../components/otelcol/otelcol.receiver.solace)
- [otelcol.receiver.splunkhec](../components/otelcol/otelcol.receiver.splunkhec)
- [otelcol.receiver.syslog](../components/otelcol/otelcol.receiver.syslog)
- [otelcol.receiver.tcplog](../components/otelcol/otelcol.receiver.tcplog)
- [otelcol.receiver.vcenter](../components/otelcol/otelcol.receiver.vcenter)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ otelcol.receiver.influxdb "influxdb_metrics" {

`otelcol.receiver.influxdb` supports the following arguments:

| Name | Type | Description | Default | Required |
| ------------------------ | -------------- | --------------------------------------------------------------- | ---------------------------------------------------------- | -------- |
| `endpoint` | `string` | `host:port` to listen for traffic on. | `"localhost:8086"` | no |
| `max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no |
| `include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no |
| `compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no |
`auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no
| Name | Type | Description | Default | Required |
| ------------------------ | -------------------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------- | -------- |
| `auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no |
| `compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no |
| `endpoint` | `string` | `host:port` to listen for traffic on. | `"localhost:8086"` | no |
| `include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no |
| `max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no |

By default, `otelcol.receiver.influxdb` listens for HTTP connections on `localhost`.
To expose the HTTP server to other machines on your network, configure `endpoint` with the IP address to listen on, or `0.0.0.0:8086` to listen on all network interfaces.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
---
canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol/otelcol.receiver.splunkhec/
description: Learn about otelcol.receiver.splunkhec
labels:
stage: public-preview
products:
- oss
title: otelcol.receiver.splunkhec
---

# `otelcol.receiver.splunkhec`

{{< docs/shared lookup="stability/public_preview.md" source="alloy" version="<ALLOY_VERSION>" >}}

`otelcol.receiver.splunkhec` accepts events in the [Splunk HEC format](https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/FormateventsforHTTPEventCollector) and forwards them to other `otelcol.*` components.
The receiver accepts data formatted as JSON HEC events under any path or as EOL separated log raw data if sent to the `raw_path` path.

{{< admonition type="note" >}}
`otelcol.receiver.splunkhec` is a wrapper over the upstream OpenTelemetry Collector `splunkhec` receiver.
Bug reports or feature requests will be redirected to the upstream repository, if necessary.
{{< /admonition >}}

You can specify multiple `otelcol.receiver.splunkhec` components by giving them different labels.

## Usage

```alloy
otelcol.receiver.splunkhec "<LABEL>" {
output {
metrics = [...]
logs = [...]
}
}
```

## Arguments

You can use the following arguments with `otelcol.receiver.splunkhec`:

| Name | Type | Description | Default | Required |
| -------------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------| ---------------------------------------------------------- | -------- |
| `access_token_passthrough` | `boolean` | If enabled perserves incomming access token as a attribute `com.splunk.hec.access_token` | `false` | no |
| `auth` | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests. | | no |
| `compression_algorithms` | `list(string)` | A list of compression algorithms the server can accept. | `["", "gzip", "zstd", "zlib", "snappy", "deflate", "lz4"]` | no |
| `endpoint` | `string` | `host:port` to listen for traffic on. | `"localhost:8088"` | no |
| `health_path` | `string` | The path reporting health checks. | `/services/collector/health` | no |
| `include_metadata` | `boolean` | Propagate incoming connection metadata to downstream consumers. | | no |
| `max_request_body_size` | `string` | Maximum request body size the server will allow. | `20MiB` | no |
| `raw_path` | `string` | The path accepting raw HEC events. Only applies when the receiver is used for logs. | `/services/collector/raw` | no |
| `splitting` | `string` | Defines the splitting strategy used by the receiver when ingesting raw events. Can be set to "line" or "none". | `"line"` | no |


By default, `otelcol.receiver.splunkhec` listens for HTTP connections on `localhost:8088`.
To expose the HTTP server to other machines on your network, configure `endpoint` with the IP address to listen on, or `0.0.0.0:8088` to listen on all network interfaces.

If `access_token_passthrough` is enabled it will be preserved as a attribute `com.splunk.hec.access_token`.
If logs or metrics are exported with `otelcol.exporter.splunkhec` it will check for this attribute and if present forward it with outgoing request.

## Blocks

You can use the following blocks with `otelcol.receiver.splunkhec`:

| Block | Description | Required |
| ---------------------------------------------------------- | -------------------------------------------------------------------------- | -------- |
| [`output`][output] | Configures where to send received telemetry data. | yes |
| [`cors`][cors] | Configures CORS for the HTTP server. | no |
| [`hec_metadata_to_otel_attrs`][hec_metadata_to_otel_attrs] | Configures OpenTelemetry attributes from HEC metadata. | no |
| [`debug_metrics`][debug_metrics] | Configures the metrics that this component generates to monitor its state. | no |
| [`tls`][tls] | Configures TLS for the HTTP server. | no |

[tls]: #tls
[cors]: #cors
[debug_metrics]: #debug_metrics
[output]: #output

### `tls`

The `tls` block configures TLS settings used for a server.
If the `tls` block isn't provided, TLS isn't used for connections to the server.

{{< docs/shared lookup="reference/components/otelcol-tls-server-block.md" source="alloy" version="<ALLOY_VERSION>" >}}

### `cors`

The `cors` block configures CORS settings for an HTTP server.

The following arguments are supported:

| Name | Type | Description | Default | Required |
| ----------------- | -------------- | -------------------------------------------------------- | ---------------------- | -------- |
| `allowed_headers` | `list(string)` | Accepted headers from CORS requests. | `["X-Requested-With"]` | no |
| `allowed_origins` | `list(string)` | Allowed values for the `Origin` header. | | no |
| `max_age` | `number` | Configures the `Access-Control-Max-Age` response header. | | no |

The `allowed_headers` specifies which headers are acceptable from a CORS request.
The following headers are always implicitly allowed:

* `Accept`
* `Accept-Language`
* `Content-Type`
* `Content-Language`

If `allowed_headers` includes `"*"`, all headers are permitted.

### `hec_metadata_to_otel_attrs`

The `hec_metadata_to_otel_attrs` block configures OpenTelemetry attributes from HEC metadata.

| Name | Type | Description | Default | Required |
| ------------ | -------- | --------------------------------------------------------------| ----------------------- |--------- |
| `host` | `string` | Specifies the mapping of the host field to a attribute. | `host.name` | no |
| `index` | `string` | Specifies the mapping of the index field to a attribute. | `com.splunk.index` | no |
| `source` | `string` | Specifies the mapping of the source field to a attribute. | `com.splunk.source` | no |
| `sourcetype` | `string` | Specifies the mapping of the sourcetype field to a attribute. | `com.splunk.sourcetype` | no |

### `debug_metrics`

{{< docs/shared lookup="reference/components/otelcol-debug-metrics-block.md" source="alloy" version="<ALLOY_VERSION>" >}}

### `output`

<span class="badge docs-labels__stage docs-labels__item">Required</span>

The `output` block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

| Name | Type | Description | Default | Required |
| --------- |--------------------------|---------------------------------------|---------|--------- |
| `logs` | `list(otelcol.Consumer)` | List of consumers to send logs to. | `[]` | no |
| `metrics` | `list(otelcol.Consumer)` | List of consumers to send metrics to. | `[]` | no |

You must specify the `output` block, but all its arguments are optional.
By default, telemetry data is dropped.
Configure the `metrics` and `logs` arguments accordingly to send telemetry data to other components.

## Exported fields

`otelcol.receiver.splunkhec` doesn't export any fields.

## Component health

`otelcol.receiver.splunkhec` is only reported as unhealthy if given an invalid configuration.

## Debug information

`otelcol.receiver.splunkhec` doesn't expose any component-specific debug information.

## Example

This example forwards received telemetry through a batch processor before finally sending it to an OTLP-capable endpoint:

```alloy
otelcol.receiver.splunkhec "default" {
output {
logs = [otelcol.processor.batch.default.input]
metrics = [otelcol.processor.batch.default.input]
}
}

otelcol.processor.batch "default" {
output {
metrics = [otelcol.exporter.otlp.default.input]
traces = [otelcol.exporter.otlp.default.input]
}
}

otelcol.exporter.otlp "default" {
client {
endpoint = sys.env("<OTLP_ENDPOINT>")
}
}
```

## Enable authentication

You can create a `otelcol.receiver.splunkhec` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.splunkhec "default" {
output {
logs = [otelcol.processor.batch.default.input]
metrics = [otelcol.processor.batch.default.input]
}
auth = otelcol.auth.basic.creds.handler
}

otelcol.auth.basic "creds" {
username = sys.env("<USERNAME>")
password = sys.env("<PASSWORD>")
}
```

<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`otelcol.receiver.splunkhec` can accept arguments from the following components:

- Components that export [OpenTelemetry `otelcol.Consumer`](../../../compatibility/#opentelemetry-otelcolconsumer-exporters)


{{< admonition type="note" >}}
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.
{{< /admonition >}}

<!-- END GENERATED COMPATIBLE COMPONENTS -->
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kafkareceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/opencensusreceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/solacereceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/splunkhecreceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/tcplogreceiver v0.122.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver v0.122.0
Expand Down Expand Up @@ -739,6 +740,7 @@ require (
github.com/oapi-codegen/runtime v1.0.0 // indirect
github.com/ohler55/ojg v1.20.1 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/extension/ackextension v0.122.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/ecsutil v0.122.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/common v0.122.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.122.0 // indirect
Expand Down
6 changes: 4 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2046,6 +2046,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexpo
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/splunkhecexporter v0.122.0/go.mod h1:q4n8s6lPiaexXu4YTwyYnDPNbE8OPAloeJaNo5aAT+g=
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/syslogexporter v0.122.0 h1:G3v1/S90iCgE6cqvmdriA+6zzZ++PEOHP6Uk2I9F/gY=
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/syslogexporter v0.122.0/go.mod h1:j2iJlR/sVR1IyNBflyGLwaIzOyx6lx8Nk48cVTWlNnY=
github.com/open-telemetry/opentelemetry-collector-contrib/extension/ackextension v0.122.0 h1:zKJUibTh/5omRXscGwCLzNJOme50l0y+E6v2Q0uhMAA=
github.com/open-telemetry/opentelemetry-collector-contrib/extension/ackextension v0.122.0/go.mod h1:OO3viI8mYkkxKKlIk6C/u6cx9StvecB43SeLy1A6B4o=
github.com/open-telemetry/opentelemetry-collector-contrib/extension/basicauthextension v0.122.0 h1:ewnxhmZGo5ugTVk715kZ8uZniek2hQRV5BzBRVTCLR0=
github.com/open-telemetry/opentelemetry-collector-contrib/extension/basicauthextension v0.122.0/go.mod h1:en/daynTJv23NQOraTOaKCyEzWcty9AN2Kyd1EWtFWM=
github.com/open-telemetry/opentelemetry-collector-contrib/extension/bearertokenauthextension v0.122.0 h1:6lgYZ6glozouz5SLENMIHRYClTYNfqhJgRE7j9KfT4s=
Expand Down Expand Up @@ -2176,6 +2178,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusrec
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.122.0/go.mod h1:6CmMa+n3XNtlKTtLXzb39+ZGVFKsx75pBnuAgef9gow=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/solacereceiver v0.122.0 h1:132lphokin3HwtEPtqLNqjYC04tmhXg9FcrXP1vvFh8=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/solacereceiver v0.122.0/go.mod h1:PHIC2G8WlOyX77+llcOgj5znBLE5kknQsaNl+dLY8OU=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/splunkhecreceiver v0.122.0 h1:ng/iPs6FXEoeEbzAyqI1jSV/xa0Ut6lqTEk2yip0xik=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/splunkhecreceiver v0.122.0/go.mod h1:opxxAzQS1j6fO+WjpeUFSUrBMtHmqLujY+UCFEwLxBU=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.122.0 h1:OqIchUstl6I4Z/hqYyDq5GznsEZjStg6Hiy9lkn1GVI=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.122.0/go.mod h1:tAfzSDDpt1ycyJeXRJseGATsJs+BNky0lNZiz4nBjjU=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/tcplogreceiver v0.122.0 h1:XUBCurUM4iH7CZKnZp/2Q6s28sC9f51hrfKwU1/H4xA=
Expand Down Expand Up @@ -3164,8 +3168,6 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.37.0 h1:1zLorHbz+LYj7MQlSf1+2tPIIgibq2eL5xkrGk6f+2c=
golang.org/x/net v0.37.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/oauth2 v0.0.0-20170807180024-9a379c6b3e95/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
Expand Down
Loading
Loading