[tlb]

The recovery process is to "only apply operations that have both intent and completion records." But then I don't see the point of logging the intent record separately. If no completion is logged, the intent is ignored. So you could log the two together.

Presumably the intent record is large (containing the key-value data) while the completion record is tiny (containing just the index of the intent record). Is the point that the completion record write is guaranteed to be atomic because it fits in a disk sector, while the intent record doesn't?

[ta8645]

It's really not clear in the article. But I _think_ the gains are to be had because you can do the in-memory updating during the time that the WAL is being written to disk (rather than waiting for it to flush before proceeding). So I'm guessing the protocol as presented, is actually missing a key step:

    Write intent record (async)
    Perform operation in memory
    Write completion record (async)
    * * Wait for intent and completion to be flushed to disk * *
    Return success to client

[gsliepen]

But this makes me wonder how it works when there are concurrent requests. What if a second thread requests data that is being written to memory by the first thread? Shouldn't it also wait for both the write intent record and completion record having been flushed to disk? Otherwise you could end up with a query that returns data that after a crash won't exist anymore.

[Manuel_D]

It's not the write ahead log that prevents that scenario, it's transaction isolation. And note that the more permissive isolation levels offered by Postgres, for example, do allow that failure mode to occur.

[Demiurge]

If thats the hypothesis, it would be good to see some numbers or proof of concept. The real world performance impact seems not that obvious to predict here.

[avinassh]

    * * Wait for intent and completion to be flushed to disk * *

if you wait for both to complete, then how it can be faster than doing a single IO?

[cbzbc]

Presumably the intent record is large (containing the key-value data) while the completion record is tiny

I don't think this is necessarily the case, because the operations may have completed in a different order to how they are recorded in the intent log.