New component: filewatch receiver

[olandr]

The purpose and use-cases of the new component

For auditing purposes there is sometimes a need to audit file operations such as creation, writing and deleting the file. The purpose of this component would be to leverage libraries such as fsbroker, fsnotify or notify, to be able to extract information on file events (CREATE, RENAME, DELETE etc.).

The component would watch a file or directory, and then send metadata and the type of action performed on the file as logs.

This component is related to #39854 where file events + contents would be sent for auditing purposes.

Other architecture that provides a similar feature:

• Logstash (auditbeat) has a File Integrity Module which is providing a similar feature, where watches can be exported downstream or exported to a sink.

Example configuration for the component

An initial version would be support configurations to specify:

• which path(s) to watch,
• which operations to consider, and
• which paths to exclude from the watch.

receivers:
  filewatch:
    path: [/var/logs/pods]
    operations: [CREATE, WRITE, REMOVE, RENAME]
    exclude_files: []

Telemetry data types supported

Logs

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

I would be interested in developing this (on behalf of SAP)! But I am neither a Member nor have a Sponsor at the moment.

[VihasMakwana]

@olandr Hello!

For new components, collector SIG meetings are great place to share your idea and get feedback from maintainers. The meeting takes place on every Wednesday. Here are more details about timings of the meetings.

[andrzej-stencel]

What do you think of the name File Watch receiver (file_watch) for the component?

What do you think of using include and exclude for the paths, in line with how File Log receiver defines match criteria?

receivers:
  file_watch:
    include:
      - /path/one/*
      - path/two.xyz
    exclude:
      - /path/one/not/*

As you already pointed out, but I want to emphasize it, this issue to re-ingest whole files on updates is closely related, as it relies on the same mechanism of watching files:

• Support complete file ingestion on update #39854

If we add a component that watches files, then maybe sending those files' contents based on changes to the files would make sense as part of this "file watch" receiver? What do you think @olandr?

[olandr]

What do you think of the name File Watch receiver (file_watch) for the component?

Going for a more generic name could make it more clear for what it is doing (rather than which lib/tool it is using).

What do you think of using include and exclude for the paths, in line with how File Log receiver defines match criteria?

Yes, that could be a good idea as well. I had a look at some other receivers (filelog esp.) for some inspiration --- will have another look. Regardless, I like the idea I will iterate the design a bit and check what makes sense!