Gelf may not work if logs are not gzipped

[danielspeixoto]

I'm using net.logstash.logback.appender.LogstashUdpSocketAppender to send logs to an logstash instance configured to parse gelf.
This implementation only send logs as plain text and not in a gzipped format. Because of that logstash may fail when parsing logs.

Being more specific, only at version 7+ logstash have support for gelf logs that are not gzipped. See logstash-plugins/logstash-input-gelf#45 for reference.

Here is an error thrown when using version 6.5.1
Gelfd failed to parse a message skipping {:exception=>#<Gelfd::UnknownHeaderError: Could not find parser for header: [123, 34]>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/gelfd-0.2.0/lib/gelfd/parser.rb:14:inparse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-gelf-3.1.1/lib/logstash/inputs/gelf.rb:212:in udp_listener'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-gelf-3.1.1/lib/logstash/inputs/gelf.rb:93:in block in run'"]}
`

When using 7+ it works properly.

[brenuart]

Thanks for the report @danielspeixoto, I was not aware of that GELF limitation in Logstash versions prior to the 7.x series. The last 6.x version was released in Aug 2019 and considering the improvements they made since then, upgrading to the newest is probably a good solution. Is that ok for you or do you have to stay on 6.x?

Correct if I'm wrong, but as I understand the documentation, the problem arises only when using UDP - TCP should be ok with plain text. Is TCP an option for you?

Note that LogstashUdpSocketAppender is a "simple" extension of Logback's legacy SyslogAppender. It does not provide any buffering and async processing as does the LogstashTcpSocketAppender appender. This means that the processing of log events (marshalling into JSON, sending UDP datagram, etc) is all done on the logging thread and may impact your application performance. If performance is a concern, then the TCP appender is a better alternative.

[danielspeixoto]

Yeah, you are right, upgrading logstash would be the best option. I'm working on that right now. Thanks