[wiradikusuma]

Materialize connects directly to event stream processors (like Kafka) --- how about Pulsar? (Goggling doesn't yield anything useful, Materialize and Pulsar are both name of brands of other things)

[chris_overseas]

Connecting Pulsar and Materialize is of interest to me too and something I might try when I find time to do so. Note that Pulsar does have a Kafka compatibility layer already[0], so it might just work out of the box. If you try this I'd be keen to hear how it goes.

[0] https://pulsar.apache.org/docs/en/adaptors-kafka/

EDIT: I don't think this adaptor will work after all, it works by replacing the Kafka Java client library with its own, so is only applicable to Java applications.

[jamii]

Adding sources is pretty easy. We just need them to provide:

* timestamped changes eg "record foo was deleted at time 42"

* watermarks eg "I've now sent you all the changes up to time 42"

Here's the implementation of the kafka source - https://github.com/MaterializeInc/materialize/blob/master/sr.... (There's also a bit of code in the parser/planner to wire up the CREATE SOURCE syntax, but it's fairly trivial.)