Logging loop

[jklap]

If FluentLogger is used as an appender with logback (such as https://github.com/sndyuk/logback-more-appenders) then any logging by FluentLogger and related classes such as RawSocketSender can end up creating a nice little logging loop.

For instance, RawSocketSender will log "Created messages" or a message when the buffer is full... all of which get routed back into slf4j and then likely routed right back into FluentLogger for logging....

While the obvious answer is to drop any org.fluentd.logger.* messages that would end up routed to the FluentLogger appender (thereby possibly leaving them routed to other appends) that would then mask any real messages but I wonder if there is a better solution?

I don't have any ideas but figured I'd throw the problem out there and maybe someone would have a good idea.

It may also be helpful to enable some type of throttling on messages--- like if the buffer is full due to a down fluentd.

Also cross posted here: sndyuk/logback-more-appenders#2

[rmelick]

Agreed. I just encountered this issue in RawSocketSender. Our log file filled up the filesystem with messages like the following:
ERROR 16-04-20_14:16:18 [pool-1-thread-1] o.f.l.s.RawSocketSender send Cannot send logs to /192.168.128.1:24224
It seems that the RawSocketSender was having issues flushing the buffer. Every log message that was sent in to fluentd (including these messages sent by RawSocketSender) would attempt to trigger another flush, which would fail and send the error again.

It seems like there could be several options to improve this. Any preferences, and I can create a pull request?

1. Throttle / rate limit the number of times RawSocketSender will print this message.
2. Move the error message into the flushBuffer() method inside the enableReconnection if statement. This would at least limit the error message to only as often as it attempts to reconnect, instead of on every log message.
3. Configure our system to explicitly filter or limit log messages from the RawSocketSender or other fluentd classes. We are actually using logback instead of directly using fluentd.

It also seems like the send() method could be simplified. It's currently calling flush() twice if the pending buffer is full. It seems like it might only need to call flush() once, and only if the pending buffer is full.

[pontello]

I am having the same problem. RawSocketSender shouldn't have a Logger instance capable of writing logs himself otherwise it will have an infinite loop.
public class RawSocketSender implements Sender { private static final Logger LOG = LoggerFactory.getLogger(RawSocketSender.class); }
As a quick solution I've downloaded RawSocketSender and removed all LOG.* calls.