New component: DNS lookup processor

[kaisecheng]

The purpose and use-cases of the new component

The DNS lookup Processor is for resolving hostnames to IP addresses and vice versa. It is particularly useful when the GeoIP processor receives a hostname and needs to convert it to an IP address in order to resolve the geo location.
The processor performs A record/CNAME record lookups to resolve hostnames to IP addresses, and PTR record lookups for reverse DNS resolution (IP to hostname). The resolved value replaces the original value.

The processor utilizes two LRU (Least Recently Used) caches with TTL (Time-to-Live): one for storing successful resolutions (hit cache) and another for storing failed lookups.

The processor can resolve hostnames/IPs using the system's default resolver, a custom hostfile, or a custom DNS server.

Example configuration for the component

procoessors:
  dnslookup:
    # A list of attributes to be resolved from hostname to IP
    resolve_attributes: []
    # A list of attributes to be reverse-looked up from IP to hostname
    reverse_attributes: []
    # Maximum number of failed resolutions to cache. Default value is `1000`. Set to `0` to disable caching
    failed_cache_size: 1000 
    # Time-to-live (in seconds) for failed resolution cache entries. Default value is `5` seconds
    failed_cache_ttl: 5
    # Maximum number of successful resolutions to cache. Default value is `1000`. Set to `0` to disable caching 
    hit_cache_size: 1000 
    # Time-to-live (in seconds) for successful resolution cache entries. Default value is `60` seconds
    hit_cache_ttl: 60 
    # Maximum number of retry attempts for DNS lookups. Default value is `2`
    max_retries: 2 
    # Timeout (in seconds) for DNS lookups. Default value is `0.5`
    timeout: 0.5
    # Path to a custom hostfile
    hostfile: "/path/to/hosts"
    # Address of a custom DNS server
    nameserver: 8.8.8.8
    # Enable system's default resolver. Default value is `false`
    enable_system_resolver: false 
    # Defines how the processor handles errors. Valid values are:
    # - `ignore`: Processor logs errors and continues to the next processor (default).
    # - `silent`: Processor ignores errors without logging and continues to the next processor.
    # - `propagate`: Processor returns errors, resulting in the payload being dropped from the collector.
    error_mode`: ignore

Resolution Order

The resolution order depends on the specified configuration and the enable_system_resolver setting:
With enable_system_resolver set to true:

1. When both hostfile and nameserver are specified:
   Custom hostfile -> Custom DNS server -> System's default resolver

2. When only nameserver is specified:
   Custom DNS server -> System's default resolver

3. When only hostfile is specified:
   Custom hostfile -> System's default resolver

4. When neither hostfile nor nameserver are specified:
   System's default resolver

With enable_system_resolver set to false, this affects all scenarios above by removing System's default resolver from the resolution order.
The processor will fail to start and throw an error If neither hostfile nor nameserver are specified and enable_system_resolver is set to false.

Considerations

• Performance impact of DNS resolutions: If each DNS lookup takes 2 milliseconds, the maximum throughput a single processor can achieve is 500 events per second.

Telemetry data types supported

Logs, metrics, traces.

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
• If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

@kaisecheng @edmocosta

Sponsor (optional)

@andrzej-stencel

Additional context

No response

[strawgate]

Could we have this support multiple nameservers right away, or is the intention to add that in a later revision? Any thoughts on TCP transport?

[kaisecheng]

Could we have this support multiple nameservers

Yes, it makes sense to support multiple.

Any thoughts on TCP transport?

I think it is a nice to have feature and could be in a later version

[andrzej-stencel]

I can sponsor this component, as long as you propose two code owners, so that one code owner's pull requests can be reviewed by the other code owner and vice versa. This will make it easier for me to review the PRs as the sponsor and third code owner.

[edmocosta]

Thank you @andrzej-stencel, I'd be glad to be the second code owner for this component. If @kaisecheng has the availability and agrees, she could be first one.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

[kaisecheng]

The proposal lacked details on how hostnames or IPs are resolved from attributes and how the results are stored. I took a lot of inspiration from GeoIP processor.

processors:
  dnslookup:
   resolve:
      enabled: true
      # Context for attributes: "resource" (default) or "record"
      context: "resource"
      # List of attributes to check. Take the first valid hostname
      attributes: ["server.address", "custom.address"]
      # Attribute to store the resolved IP
      lookedup_attribute: "server.ip"
   reverse:
      enabled: false
      context: "resource"
      # List of attributes to check. Take the first valid IP
      attributes: ["server.ip", "custom.ip"]
      # Attribute to store the resolved hostname
      lookedup_attribute: "server.address"
   miss_cache_size: 1000
   miss_cache_ttl: 30
   hit_cache_size: 1000
   hit_cache_ttl: 60
   max_retries: 2
   timeout: 0.5
   hostfiles: ["/path/to/hosts"]
   nameservers: [ "8.8.8.8" ]
   enable_system_resolver: false 

The processor iterates through attributes and uses the first valid hostname or IP. The context specifies whether to look in resource attributes or within a telemetry record (such as a span, log, or data point). The result of the resolution is stored in the attribute specified by lookedup_attribute.

Both forward (resolve) and reverse DNS lookups share the same caching, retry, and timeout configurations.