manage/reduce bloat

[cforce]

Component(s)

No response

Describe the issue you're reporting

Trying to install otelcol on a resource limited device.
With some modules i already go above 149 MB in size.

v121

dist:
  name: otecol
  version: ${OTELCOL_VERSION}
  include_core: true
  output_path: bin/collector
  debug_compilation: false

receivers:
  - gomod: go.opentelemetry.io/collector/receiver/nopreceiver v${OTELCOL_VERSION} # stable
  - gomod: go.opentelemetry.io/collector/receiver/otlpreceiver v${OTELCOL_VERSION}
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v${OTELCOL_VERSION} # beta
    # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/prometheusreceiver
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v${OTELCOL_VERSION} # alpha
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/podmanreceiver v${OTELCOL_VERSION} # development    
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver v${OTELCOL_VERSION} # alpha  

processors:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v${OTELCOL_VERSION}    
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/metricstransformprocessor v${OTELCOL_VERSION} # alpha
    - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/cumulativetodeltaprocessor v${OTELCOL_VERSION} # beta
  - gomod: go.opentelemetry.io/collector/processor/batchprocessor v${OTELCOL_VERSION} # stable
  - gomod: go.opentelemetry.io/collector/processor/memorylimiterprocessor v${OTELCOL_VERSION} # stable
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/tailsamplingprocessor v${OTELCOL_VERSION}
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/probabilisticsamplerprocessor v${OTELCOL_VERSION}
  - gomod: github.com/observiq/bindplane-otel-collector/processor/snapshotprocessor v1.72.0

exporters:
  - gomod: go.opentelemetry.io/collector/exporter/nopexporter v${OTELCOL_VERSION} # stable  
  - gomod: go.opentelemetry.io/collector/exporter/otlpexporter v${OTELCOL_VERSION} # stable
  - gomod: go.opentelemetry.io/collector/exporter/otlphttpexporter v${OTELCOL_VERSION} # stable
  - gomod: go.opentelemetry.io/collector/exporter/debugexporter v${OTELCOL_VERSION} # stable

connectors:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/connector/spanmetricsconnector v${OTELCOL_VERSION} # alpha  
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector v${OTELCOL_VERSION} # dev
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/connector/exceptionsconnector v${OTELCOL_VERSION} # dev

extensions:
  - gomod: go.opentelemetry.io/collector/extension/zpagesextension v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/healthcheckextension v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/oauth2clientauthextension v${OTELCOL_VERSION} # beta
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/filestorage v${OTELCOL_VERSION}  # stable
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension v${OTELCOL_VERSION} # alpha
  - gomod: github.com/observiq/bindplane-otel-collector/extension/bindplaneextension v1.72.0

providers:
  - gomod: go.opentelemetry.io/collector/confmap/provider/envprovider v1.27.1-0.20250307194215-7d3e03e500b0
  - gomod: go.opentelemetry.io/collector/confmap/provider/fileprovider v1.27.1-0.20250307194215-7d3e03e500b0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpprovider v1.27.1-0.20250307194215-7d3e03e500b0
  - gomod: go.opentelemetry.io/collector/confmap/provider/httpsprovider v1.27.1-0.20250307194215-7d3e03e500b0
  - gomod: go.opentelemetry.io/collector/confmap/provider/yamlprovider v1.27.1-0.20250307194215-7d3e03e500b0

I wanted to find out which modules bloat top down and installed bloaty and executed against the otecol exe

I was surprised about the result as it seem most from the weight comes from the core, doesn' it?

    FILE SIZE        VM SIZE
 --------------  --------------
  34.0%  64.8Mi  34.8%  52.1Mi    [1591 Others]
  19.2%  36.5Mi  24.4%  36.5Mi    [section .gopclntab]
  14.9%  28.4Mi  19.0%  28.4Mi    [section .rodata]
   7.7%  14.6Mi   8.8%  13.2Mi    github.com/aws/aws-sdk-go-v2/service/ec2
   5.7%  10.9Mi   0.0%       0    [section .debug_info]
   3.2%  6.04Mi   2.7%  4.08Mi    github.com/aws/aws-sdk-go/service/ec2
   2.6%  4.95Mi   2.7%  4.01Mi    google.golang.org/api/compute/v1
   1.9%  3.70Mi   0.0%       0    [section .debug_line]
   1.7%  3.19Mi   1.9%  2.79Mi    k8s.io/api/core/v1
   1.2%  2.33Mi   0.0%       0    [section .debug_loc]
   1.2%  2.28Mi   0.0%       0    [section .strtab]
   1.1%  2.08Mi   1.0%  1.48Mi    github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter/filterottl
   1.0%  1.90Mi   0.0%       0    [section .debug_frame]
   0.9%  1.66Mi   0.8%  1.27Mi    github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5
   0.7%  1.40Mi   0.9%  1.38Mi    github.com/aws/aws-sdk-go/aws/endpoints
   0.6%  1.21Mi   0.5%   824Ki    github.com/aws/aws-sdk-go/service/lightsail
   0.6%  1.15Mi   0.5%   815Ki    github.com/envoyproxy/go-control-plane/envoy/config/core/v3
   0.5%   965Ki   0.5%   766Ki    github.com/open-telemetry/opentelemetry-collector-contrib/connector/routingconnector
   0.4%   841Ki   0.4%   670Ki    github.com/linode/linodego
   0.4%   835Ki   0.5%   835Ki    [section .data]
   0.4%   820Ki   0.5%   721Ki    net/http
 100.0%   190Mi 100.0%   149Mi    TOTAL

Is there an easier way isnetad of using bloaty to detect wwich go modules bloat. Are there any ways to reduce site without dropping oteocol recivers, expoeeter, processor, etc

What about compression like upx

As i compile with ocb default , which shall not add symbols(oc by default does go build -ldflags "-s -w" ) , i don't have an idea why its so big . As well other sources document the huge size

[atoulme]

I don't see that core contributes to bloat the most. You can open an issue under core for the builder to support upx, and you can open issues under opentelemetry-collector-releases to start measuring and tracking file size over time.

I don't know how to help further ; more folks involved in golang might have tips for you.

[cforce]

Thanks for the clarification. I might have misunderstood how to interpret the output, or perhaps I'm not using the right tool for this analysis. In the first four lines of the report, there’s clearly some relevant size shown — but I’m unclear on what exactly falls under “others.” I initially assumed those top entries (or at least some of them) belonged to core components, which led to the assumption that core was contributing significantly to the bloat. If that’s incorrect, I’d appreciate any insight into what those entries actually represent and how best to attribute them.

[nozomi-gedimitr]

In a previous situation before, we found out that the resourcedetectionprocessor processor has a very big footprint: with its capabilities to extract information from different setups / clouds, it is bringing lots of dependencies. The bloaty report is also hinting that way, mentioning those cloud related packages.

Try making a build without this processor and see if it will make a difference