You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a user I would like to be able to easily drop all fields in a message that are either empty or otherwise not useful in order to reduce storage of unused fields. A common use case is an empty field being denoted by a - value. Ideally, a single rule could be used to drop any fields in a message with this value. For example:
rule "drop fields with - as content"
when
true
then
remove_fields_by_content("-");
end
Why?
Currently, this can be accomplished on a field by field basis with many rules that looks like this:
rule "drop field my_field if empty"
when
to_string($message.my_field) == "-"
then
remove_single_field("my_field");
end
Supporting this in a single pipeline rule would make this cleanup much more efficient.
I think we ought to have the @Graylog2/architecture team weigh in on this. I imagine it's more performant to do this in Java than to do it by having a pipeline rule for each possible field in a message, but I expect it still has the potential to be a performance concern to iterate over all of the fields in a message checking to see if they match a particular value.
@waab76 agreed. Just made the issue for tracking discussion and wrote the code because it was a quick copy/tweak of the existing remove_multiple_fields rule.
What?
As a user I would like to be able to easily drop all fields in a message that are either empty or otherwise not useful in order to reduce storage of unused fields. A common use case is an empty field being denoted by a
-
value. Ideally, a single rule could be used to drop any fields in a message with this value. For example:Why?
Currently, this can be accomplished on a field by field basis with many rules that looks like this:
Supporting this in a single pipeline rule would make this cleanup much more efficient.
See https://github.com/Graylog2/support/issues/202 for more context.
The text was updated successfully, but these errors were encountered: