[Spivak]

I feel like the inventory thing is a bit of a straw-man because the situation is set out in such a way that you need transactions for it to work. If you find yourself wishing you had a global write-lock on a topic to then of course it won't work. Modeling your data for Kafka is work just the same as it is for MySQL. Of course it might not be the best tool for the job but you should at least give it a fair shake.

You should be able to post "buy" messages to a topic without fear that it messes up your data integrity. Who cares if two people are fighting over the last item? You have a durable log. Post both "buys" and wait for the "confirm" message from a consumer that's reading the log at that point in time, validates, and confirms or rejects the buys. At the point that the buy reaches a consumer there is enough information to know for sure whether it's valid or not. Both of the buy events happened and should be recorded whether they can be fulfilled or not.

[jacques_chester]

> Who cares if two people are fighting over the last item?

The two people, at least. Customers tend to be a bit underwhelmed by "well, the CAP theorem..." as a customer service script.

[Spivak]

None of this shows up as user-facing any differently than a relational database. No CAP theorem at all.

Kafka:

User clicks buy and it shows “processing” which behind the scenes posts the buy message and waits for a “confirmed” message. When it’s confirmed user is directed to success! If someone else posts the buy before them they get back a “failed: sold out” message.

Relational:

User clicks buy and it shows “processing” which behind the scenes tries to get a lock on the db, looks at inventory, updates it if there’s still one available, and creates a row in the purchases table. If all this works the user is directed to success. If by the time the lock was acquired the inventory was zero the server returns “failure: sold out”.

[jacques_chester]

The CAP theorem line was smart-arsery.

The thing here is that the database can update the cart and the inventory in one logical step, to the exclusion of others.

The Kafka approach doesn't guarantee that out of the box, leading to the creation of de facto locking protocols (write cart intent, read cart intent, write inventory intent ...). A traditional database does that for you with selectable levels of guarantees.

[Spivak]

You're absolutely right but I think the advantage is that to the users of the database it doesn't really feel like there's any locking going on and it's this implicit thing that's constructed from the history.

Like for a hypothetical ticket selling platform you just get a log like

    00:00 RESERVE AAA FOR xxx, 5 min
    00:02 BUY     BBB FOR qqq
    00:15 RESERVE AAA FOR yyy, 5 min
    00:23 CONFIRM AAA FOR xxx
    00:25 CONFIRM BBB for qqq
    00:27 REJECT  AAA FOR yyy, "already reserved"
    05:16 RESERVE AAA FOR zzz, 5 min
    05:34 CONFIRM AAA FOR zzz

So although there's "locking" going on here enforced by the consumer sending the confirms the producer just sends intents/events and sees whether they're successful or not and both producer and consumer are stateless.

I guess it just depends on which model you think is more a PITA to work with.

[AndyPa32]

Yeah, if you are lucky enough that your cart and inventory is stored in the same database.

[jacques_chester]

Of course. That's why Materialize are interesting, they are apparently able to give a greatly improved compute:result ratio than the previous state of the art.