filelog/syslog receiver and exporter problem.

[recep-instructor]

Component(s)

exporter/syslog

Describe the issue you're reporting

Hi,

I have been working on a project about OpenTelemetry Collector, Loki and Grafana for a long time. I have 1 k8s cluster (1 control node and 3 worker nodes). I have deployed Loki, Grafana and OpenTelemetry/Otel-Collector with Helm in this cluster. My aim is to send syslogs of k8s cluster to an external syslog server and to observe them from Grafana, and also to monitor logs of pods in k8s cluster from Grafana dashboard. I have been working on it for 10 days but I have some problems that I can't solve, one of them is regex problem. Logs are sent to Loki with Filelog/Syslog receiver, I can see them in Grafana, but logs are sent to external syslog server empty. I need urgent help on this issue, I would be grateful if a friend can help me. I am sharing my Otel-Collector-values.yml file.

[github-actions]

Pinging code owners:

• exporter/syslog: @kasia-kujawa @rnishtala-sumo @andrzej-stencel

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[recep-instructor]

otel-collector-values-apprecode.txt and output of syslog server were attached.

otel-collector-values-apprecode.txt

[andrzej-stencel]

Hey @recep-instructor, I think I understand where the problem is. You are reading the syslog logs from files like /var/log/syslog with the File Log receiver without further processing, which means the contents of each line lands in the Body of the log message. After that, you are sending those logs with the Syslog exporter to a syslog server, but the Syslog exporter sends a mostly empty syslog message, because it tries to read the syslog entries' contents from the log attributes, and not from the Body. For example, the syslog host is expected to be available in "hostname" attribute, and the actual syslog message in the "message" log attribute.

To fix that, you need to transform the logs so that they're understandable by the Syslog exporter. See the expected attributes here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.126.0/exporter/syslogexporter/README.md#rfc5424.

I would suggest to use the dedicated syslog parser, but I'm afraid it's no good for parsing syslog logs from the files, as they miss the header with the priority and version. Due to that, I recommend to configure the regex parser with a custom regex to set the right attributes. Here's a simple example:

    filelog/syslog:
      include:
        - /var/log/syslog
        - /var/log/auth.log
        - /var/log/messages
        - /var/log/kern.log
      start_at: beginning
      include_file_path: true
      include_file_name: true
      operators:
        - type: add
          id: add-syslog-source
          field: resource["log.source"]
          value: "syslog"
        - type: regex_parser
          id: custom-syslog-parser
          regex: '^(?P<timestamp>\S+) (?P<hostname>[^ ]+) (?P<appname>[^ ]+) (?P<message>.*)$'
          timestamp:
            parse_from: attributes.timestamp
            layout: "%Y-%m-%dT%H:%M:%S.%f%j"

I've added the custom-syslog-parser operator to your configuration of the filelog/syslog receiver.

It does seem to work on the logs in my /var/log/syslog file, which looks something like this:

2025-05-20T14:22:42.978625+02:00 astencel rtkit-daemon[2569]: Supervising 9 threads of 6 processes of 1 users.

If the format in your syslog files is different, you might need to tweak the regex. Here's the docs for the regex_parser operator and for timestamp parsing.

Let me know if this helps and any further questions.

[recep-instructor]

I appreciate you, I will try and inform you as soon as possible