I'm confused with Exit_On_Eof

[belmarkjunecaday]

I have multiple processes that writes to multiple files which fluent-bit will collect from. These are daily logs, around 3k files, some can reach up to 700MB, some as small as 3kb. I always have to restart fluent-bit because at some point it stops collecting and when I look into mdsd.err file I have this error message

AMACoreAgent connector socket Data:2 read handler error code 2; End of file

I tried to set the Exit_On_Eof to true. What happens when fluent-bit is reading a file while my application is still writing to that same file? will fluent-bit wait for the process to done writing and consider it as end of file? or will I lose some log data if this happens?

Here is my config:

[SERVICE]
    flush                     1
    daemon                    Off
    log_level                 info
    log_file                  /var/opt/microsoft/azuremonitoragent/log/fluentbit.log
    storage.path              /etc/opt/microsoft/azuremonitoragent/config-cache/fluentbit/stg
    storage.checksum          off
    storage.path              /shares/log/azure-log-analytics-storage
    storage.backlog.mem_limit 100M
    storage.max_chunks_up     50M
    storage.sync              full
    http_server               Off
    parsers_file              /etc/opt/microsoft/azuremonitoragent/config-cache/fluentbit/parsers.conf
[INPUT]
    Name                 tail
    Path                 /log/2025*/*.log, /log/2025*/*.txt, /log/2025*/*cm
    Exit_On_Eof          true
    storage.type          filesystem
    Ignore_Older         1h
    threaded             true
    tag                  c18143338653385411927_12174608305570740215
    multiline.parser     timestamp-format1
    Path_Key             FilePath
    key                  RawData
    Mem_Buf_Limit        1024MB
    Buffer_Chunk_Size    50MB
    Buffer_Max_Size      1024MB
    Skip_Long_Lines      On
    Skip_Empty_Lines     On
    Refresh_Interval     10
[OUTPUT]
    Name                       forward
    Match                      *
    Host                       127.0.0.1
    Port    28230    Time_as_Integer            true

[patrick-stephens]

Note Fluent Bit will exit if EOF is hit on any of those files - it will not wait for them all to be EOF.