[frsyuki]

ACK of network transfer is available ("require_ack_response" option). This option ends up choice of at-most-once semantics vs. at-least-once semantics. You need to choose and you can choose.

Fuentd provides "buffer_type file" to buffer records on disk. Shutting down won't loose data. If you need to choose memory buffer for performance reasons, fluentd enables "flush_at_shutdown" option by default.

You would also want to use <secondary> feature. This lets you to write a buffer chunk to another storage if the primary destination is not available "retry_limit" times.

Those concerns would be solved by the document: http://docs.fluentd.org/articles/out_forward#buffered-output...

[core0]

Ah, thanks - require_ack_response sounds like what I was missing. Some blogs are from before this was added in 0.12 so I didn't know about it.

I am still interested in forwarder failure cases - I have replied to kiyoto's comment talking about the HA docs, which still talk about some other cases that can lose messages.

In this case:

* The process dies immediately after receiving the events, but before writing them into the buffer.

Is it possible to require acknowledgement that the log event has been written to the buffer? Is that separate to what require_ack_response does?