[cicloid]

Did you consider using Docker for the provisioning of Kafka?

A couple of days ago I remember reading how difficult was to deploy Oracle on Linux and how Docker made this a breeze. I wonder if Kafka would also fall onto the same premise.

[josephg]

Probably. I was bullish on docker in the past, but I'm no longer convinced its worth the trouble for small projects. It adds an awful lot of operational complexity for what is essentially a more complex abstraction around processes.

I think its a nice tool for deployment and making reproducible builds, but a lot of other things become harder through docker - like managing a databases's data, and communication between local processes.

Maybe the tooling has improved in the last few years, but I've gone back to the raw unix coalface.

[Drdrdrq]

>... but a lot of other things become harder through docker - like managing a databases's data, and communication between local processes.

It doesn't have to be this way. If you use shared folders to persist data on host you are in no worse position than you would be in if you used natively installed app, persistence wise.

I think the Docker's focus on orchestration (which makes business sense for them) is the reason why running DBs in containers got bad reputation. But really, if you use shared dirs with host and view containers as processes you can use them for DBs too.

IPC with containers OTOH forces you to architect the system as a bunch of microservices, which is usually not a bad idea either.