Skip to content

Introduce eBPF Profiler distribution #908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Apr 28, 2025
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
2c4186a
add new ebpf-profiler distribution
dmathieu Apr 10, 2025
5ef69ef
add changelog entry
dmathieu Apr 10, 2025
b7ac411
mark ebpf-profiler as a valid component
dmathieu Apr 10, 2025
c23f474
only release on amd for now
dmathieu Apr 10, 2025
0e909d4
enable cgo
dmathieu Apr 10, 2025
2011d82
don't rely on scratch image
dmathieu Apr 10, 2025
a570bf3
use a custom config file for profiler
dmathieu Apr 10, 2025
2195474
run tests with profiles feature flags
dmathieu Apr 10, 2025
2dabd17
run ebpf container privileged
dmathieu Apr 10, 2025
0440c63
run container as root
dmathieu Apr 10, 2025
ecac474
mount debugfs?
dmathieu Apr 10, 2025
f990b84
add dash to org name
dmathieu Apr 10, 2025
f0f6315
add ebpf-profiler exception for components to include
dmathieu Apr 10, 2025
75ca5b7
it's connector and extensions that must facilitate collection and pro…
dmathieu Apr 10, 2025
fe7f8ef
Update distributions/otelcol-ebpf-profiler/README.md
dmathieu Apr 11, 2025
5449643
Update .github/workflows/release-ebpf-profiler.yaml
dmathieu Apr 11, 2025
850add7
add arm64
dmathieu Apr 11, 2025
2c03cf9
remove unneeded name
dmathieu Apr 11, 2025
b253827
Merge branch 'main' into otelcol-ebpf-profiler
mowies Apr 14, 2025
d70c4b5
Update .github/workflows/ci-goreleaser-ebpf-profiler.yaml
dmathieu Apr 15, 2025
e28619c
add extension to config_file parameter
dmathieu Apr 15, 2025
3a9b8a7
move the config option to be after all the docker ones
dmathieu Apr 15, 2025
b62ed31
remove arm64
dmathieu Apr 15, 2025
0ca70e6
Merge branch 'main' into otelcol-ebpf-profiler
dmathieu Apr 15, 2025
7259f10
Merge branch 'main' into otelcol-ebpf-profiler
dmathieu Apr 15, 2025
4818355
use custom ld flags
dmathieu Apr 15, 2025
85ded16
add custom go tags
dmathieu Apr 15, 2025
ca632d1
only keep the extldflags ldflag
dmathieu Apr 15, 2025
2331c4f
Merge branch 'main' into otelcol-ebpf-profiler
dmathieu Apr 18, 2025
c6db5e0
remove unneeded arm version
dmathieu Apr 22, 2025
defc5b2
document the need for cgo
dmathieu Apr 22, 2025
3230c64
don't release ebpf images yet
dmathieu Apr 22, 2025
7da8a63
refine cgo doc
dmathieu Apr 22, 2025
5278351
Merge branch 'main' into otelcol-ebpf-profiler
dmathieu Apr 22, 2025
7528499
remove arm64 from goreleaser
dmathieu Apr 23, 2025
946d131
add glibc version requirement
dmathieu Apr 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .chloggen/ebpf-profiler.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: new_component

# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
component: otelcol-ebpf-profiler

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Introduce eBPF Profiler Distribution

# One or more tracking issues or pull requests related to the change
issues: [908]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
6 changes: 3 additions & 3 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
#
# Learn about membership in OpenTelemetry community:
# https://github.com/open-telemetry/community/blob/main/community-membership.md
#
#
# Learn about CODEOWNERS file format:
#
# Learn about CODEOWNERS file format:
# https://help.github.com/en/articles/about-code-owners
#

Expand All @@ -18,4 +18,4 @@
/distributions/otelcol-otlp/manifest.yaml @open-telemetry/collector-releases-approvers @open-telemetry/collector-approvers
/distributions/otelcol-contrib/manifest.yaml @open-telemetry/collector-releases-approvers @open-telemetry/collector-contrib-approvers
/distributions/otelcol-k8s/manifest.yaml @open-telemetry/collector-releases-approvers @open-telemetry/helm-approvers @open-telemetry/operator-approvers

/distributions/otelcol-ebpf-profiler @open-telemetry/collector-releases-approvers @open-telemetry/ebpf-profiler-approvers
19 changes: 15 additions & 4 deletions .github/workflows/base-ci-goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,16 @@ on:
type: boolean
default: false
description: "Set to true to fetch latest otelcol-contrib main branch version instead of building the version in this repo"
config_file:
required: false
type: string
default: default-config
docker_run_options:
required: false
type: string
otelcol_run_options:
required: false
type: string

env:
# renovate: datasource=github-tags depName=goreleaser-pro packageName=goreleaser/goreleaser-pro
Expand Down Expand Up @@ -197,12 +207,12 @@ jobs:
# Find version number and types of built artifacts
echo "version=$(cat ./distributions/${{ inputs.distribution }}/dist/**/metadata.json | jq -r '.version')" >> "$GITHUB_OUTPUT"
echo "types=$(cat ./distributions/${{ inputs.distribution }}/dist/**/artifacts.json | jq '[.[].type]' | tr -d '\n' )" >> "$GITHUB_OUTPUT"

# Find GOOS and GOARCH
os=$(cat ./distributions/${{ inputs.distribution }}/dist/**/artifacts.json | jq -r '.[] | select(.name | contains(".tar.gz")) | .goos')
arch=$(cat ./distributions/${{ inputs.distribution }}/dist/**/artifacts.json | jq -r '.[] | select(.name | contains(".tar.gz")) | .goarch')
echo "arch=$os-$arch" >> "$GITHUB_OUTPUT"

# Find binary path
echo "binary-path=$(cat ./distributions/${{ inputs.distribution }}/dist/**/artifacts.json | jq -r 'map(select(any(.type; contains("Binary")))) | .[].path' )" >> "$GITHUB_OUTPUT"

Expand Down Expand Up @@ -276,9 +286,10 @@ jobs:
echo "Running container..."
docker run --name ${{ inputs.distribution }} \
-d \
-v ${PWD}/tests/docker-tests/default-config.yaml:/config.yaml \
${{ inputs.docker_run_options }} \
-v ${PWD}/tests/docker-tests/${{ inputs.config_file }}.yaml:/config.yaml \
${{ inputs.distribution }}:${{ needs.check-goreleaser.outputs.version }}-${{ matrix.GOOS }}-${{ matrix.GOARCH }} \
--config /config.yaml
--config /config.yaml ${{ inputs.otelcol_run_options }}
echo "docker ps:"
echo ""
docker ps
Expand Down
37 changes: 37 additions & 0 deletions .github/workflows/ci-goreleaser-ebpf-profiler.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: CI - eBPF Profiler - GoReleaser

on:
merge_group:
push:
branches: [main]
paths:
- "distributions/otelcol-ebpf-profiler/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"
pull_request:
branches: [main]
paths:
- "distributions/otelcol-ebpf-profiler/**"
- "cmd/**"
- ".github/**"
- "scripts/**"
- "Makefile"
- "go.mod"
- "go.sum"

jobs:
check-goreleaser:
name: CI - eBPF Profiler - GoReleaser
uses: ./.github/workflows/base-ci-goreleaser.yaml
with:
distribution: otelcol-ebpf-profiler
config_file: ebpf-profiler-config
docker_run_options: '--privileged -v /sys/kernel/debug/:/sys/kernel/debug/'
otelcol_run_options: '--feature-gates=service.profilesSupport'
goos: '[ "linux" ]'
goarch: '[ "amd64", "arm64" ]'
secrets: inherit
16 changes: 16 additions & 0 deletions .github/workflows/release-ebpf-profiler.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Release eBPF Profiler

on:
push:
tags: ["v*"]

jobs:
release:
name: Release eBPF Profiler
uses: ./.github/workflows/base-release.yaml
with:
distribution: otelcol-ebpf-profiler
goos: '[ "linux" ]'
goarch: '[ "amd64", "arm64" ]'
secrets: inherit
permissions: write-all
7 changes: 4 additions & 3 deletions .github/workflows/scripts/bump-versions.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/bin/bash
set -e
# This script reads current versions and takes optional next versions, and updates the
# version in the specified files. If next version is not provided, it will infer the
# next semantic version (e.g. v0.110.0 -> v0.111.0 or v1.16.0 -> v1.17.0) based on the
# version in the specified files. If next version is not provided, it will infer the
# next semantic version (e.g. v0.110.0 -> v0.111.0 or v1.16.0 -> v1.17.0) based on the
# current version(s) read in.

# List of files to update
Expand All @@ -11,6 +11,7 @@ manifest_files=(
"distributions/otelcol/manifest.yaml"
"distributions/otelcol-k8s/manifest.yaml"
"distributions/otelcol-otlp/manifest.yaml"
"distributions/otelcol-ebpf-profiler/manifest.yaml"
)

# Function to display usage
Expand Down Expand Up @@ -206,7 +207,7 @@ create_pr() {

gh pr create --title "[chore] Prepare release $next_version" \
--body "This PR updates the version from $current_version to $next_version" \
--base main --head "$branch_name" --draft
--base main --head "$branch_name" --draft
}

# TODO: Once Collector 1.0 is released, we can consider removing the
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ TOOLS_BIN_NAMES := $(addprefix $(TOOLS_BIN_DIR)/, $(notdir $(shell echo $(TOOLS_
CHLOGGEN := $(TOOLS_BIN_DIR)/chloggen
CHLOGGEN_CONFIG := .chloggen/config.yaml

DISTRIBUTIONS ?= "otelcol,otelcol-contrib,otelcol-k8s,otelcol-otlp"
DISTRIBUTIONS ?= "otelcol,otelcol-contrib,otelcol-k8s,otelcol-otlp,otelcol-ebpf-profiler"

ci: check build
check: ensure-goreleaser-up-to-date validate-components
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Current list of distributions:
- [OpenTelemetry Collector Contrib (also known as "otelcol-contrib")](./distributions/otelcol-contrib)
- [OpenTelemetry Collector for Kubernetes (also known as "otelcol-k8s")](./distributions/otelcol-k8s)
- [OpenTelemetry Collector OTLP (also known as "otelcol-otlp")](./distributions/otelcol-otlp)
- [OpenTelemetry Collector eBPF Profiler (also known as "otelcol-ebpf-profiler")](./distributions/otelcol-ebpf-profiler)

## Community

Expand Down
34 changes: 26 additions & 8 deletions cmd/goreleaser/internal/configure.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,15 @@ import (
)

const (
armArch = "arm"
coreDistro = "otelcol"
contribDistro = "otelcol-contrib"
k8sDistro = "otelcol-k8s"
otlpDistro = "otelcol-otlp"
ghcr = "ghcr.io/open-telemetry/opentelemetry-collector-releases"
binaryNamePrefix = "otelcol"
imageNamePrefix = "opentelemetry-collector"
armArch = "arm"
coreDistro = "otelcol"
contribDistro = "otelcol-contrib"
k8sDistro = "otelcol-k8s"
otlpDistro = "otelcol-otlp"
ebpfProfilerDistro = "otelcol-ebpf-profiler"
ghcr = "ghcr.io/open-telemetry/opentelemetry-collector-releases"
binaryNamePrefix = "otelcol"
imageNamePrefix = "opentelemetry-collector"
)

var (
Expand All @@ -45,6 +46,7 @@ var (
winContainerArchs = []string{"amd64"}
darwinArchs = []string{"amd64", "arm64"}
k8sArchs = []string{"amd64", "arm64", "ppc64le", "s390x"}
ebpfProfilerArchs = []string{"amd64", "arm64"}

imageRepos = []string{ghcr}

Expand Down Expand Up @@ -141,6 +143,20 @@ var (
newContainerImageManifests(d.name, "linux", k8sArchs, containerImageOptions{}),
)
}).WithDefaultArchives().WithDefaultChecksum().WithDefaultSigns().WithDefaultDockerSigns().WithDefaultSBOMs().Build()

// ebpf-profiler distro
ebpfProfilerDist = newDistributionBuilder(ebpfProfilerDistro).WithConfigFunc(func(d *distribution) {
d.buildConfigs = []buildConfig{
&fullBuildConfig{targetOS: "linux", targetArch: ebpfProfilerArchs},
}
d.containerImages = slices.Concat(
newContainerImages(d.name, "linux", ebpfProfilerArchs, containerImageOptions{armVersion: "7"}),
)
d.containerImageManifests = slices.Concat(
newContainerImageManifests(d.name, "linux", ebpfProfilerArchs, containerImageOptions{}),
)
d.enableCgo = true
}).WithDefaultArchives().WithDefaultChecksum().WithDefaultSigns().WithDefaultDockerSigns().WithDefaultSBOMs().Build()
)

type buildConfig interface {
Expand Down Expand Up @@ -600,6 +616,8 @@ func BuildDist(dist string, onlyBuild bool) config.Project {
return otlpDist.BuildProject()
case k8sDistro:
return k8sDist.BuildProject()
case ebpfProfilerDistro:
return ebpfProfilerDist.BuildProject()
case contribDistro:
if onlyBuild {
return contribBuildOnlyDist.BuildProject()
Expand Down
96 changes: 96 additions & 0 deletions distributions/otelcol-ebpf-profiler/.goreleaser.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
version: 2
project_name: opentelemetry-collector-releases
env:
- COSIGN_YES=true
- LD_FLAGS=-s -w
- BUILD_FLAGS=-trimpath
release:
replace_existing_artifacts: true
builds:
- id: otelcol-ebpf-profiler-linux
goos:
- linux
goarch:
- amd64
- arm64
dir: _build
binary: otelcol-ebpf-profiler
ldflags:
- '{{ .Env.LD_FLAGS }}'
flags:
- '{{ .Env.BUILD_FLAGS }}'
archives:
- id: otelcol-ebpf-profiler
builds:
- otelcol-ebpf-profiler-linux
name_template: '{{ .Binary }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}{{ if .Mips }}_{{ .Mips }}{{ end }}'
checksum:
name_template: '{{ .ProjectName }}_otelcol-ebpf-profiler_checksums.txt'
dockers:
- goos: linux
goarch: amd64
dockerfile: Dockerfile
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:{{ .Version }}-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:latest-amd64
build_flag_templates:
- --pull
- --platform=linux/amd64
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
- --label=org.opencontainers.image.licenses=Apache-2.0
use: buildx
- goos: linux
goarch: arm64
dockerfile: Dockerfile
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:{{ .Version }}-arm64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:latest-arm64
build_flag_templates:
- --pull
- --platform=linux/arm64
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.name={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- --label=org.opencontainers.image.source={{.GitURL}}
- --label=org.opencontainers.image.licenses=Apache-2.0
use: buildx
docker_manifests:
- name_template: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:{{ .Version }}
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:{{ .Version }}-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:{{ .Version }}-arm64
- name_template: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:latest
image_templates:
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:latest-amd64
- ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-ebpf-profiler:latest-arm64
signs:
- cmd: cosign
args:
- sign-blob
- --output-signature
- ${artifact}.sig
- --output-certificate
- ${artifact}.pem
- ${artifact}
signature: ${artifact}.sig
artifacts: all
certificate: ${artifact}.pem
docker_signs:
- args:
- sign
- ${artifact}
artifacts: all
sboms:
- id: archive
artifacts: archive
- id: package
artifacts: package
monorepo:
tag_prefix: v
partial:
by: target
4 changes: 4 additions & 0 deletions distributions/otelcol-ebpf-profiler/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM golang:1.23.8

COPY --chmod=755 otelcol-ebpf-profiler /otelcol-ebpf-profiler
ENTRYPOINT ["/otelcol-ebpf-profiler"]
24 changes: 24 additions & 0 deletions distributions/otelcol-ebpf-profiler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# OpenTelemetry Collector eBPF Profiling Distribution

This distribution is made specifically to be used as a node agent to gather
profiles on all processes running on the system.

It contains the [eBPF profiler
receiver](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) as
well as a subset of components from [OpenTelemetry Collector
Core](https://github.com/open-telemetry/opentelemetry-collector) and
[OpenTelemetry Collector
Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib).

## Components

The full list of components is available in the [manifest](manifest.yaml).

### Rules for Component Inclusion

- Only includes components from Contrib and Core, except for the ebpf-profiler receiver.
- Only components that are Alpha or higher.
- All processors, connectors, and extensions must facilitate the collection and processing of data that is generated by the eBPF profiler.
- All components must be vendor-neutral.
- Only exporters that use OTLP are allowed.
- To facilitate troubleshooting, the nop, debug, and file exporters are exceptions.
Loading
Loading