[dflock]

Have an interface for a queuing system and support other things, not just Kafka. Ideally, you want a default/dev instance to ship with something super simple, zero setup and maybe in-memory - but allowing you to swap-in kafka or something more capable as needed.

[pranay01]

That's an interesting point. Curious, would you use a project which supports a simple/in-memory datastore, but not anything which would be useful in production environment? Do you think that easy to get running and setup in dev environment valuable for adoption - even if it won't work in prod?

I am trying to understand - what would be a good way to prioritise.

[dflock]

> Do you think that easy to get running and setup in dev environment valuable for adoption

Yes, it'll make a big difference to adoption. If step one of your setup instructions are "provision a Kafka cluster", then you are going to lose 90% of people right there.

Ideally, your dev install is super simple and has a built-in in-memory queue thing. The key here, is to make it as simple as possible to get started. Once people have tried it out, and become invested in you, then you can say "for production scale, use Kafka instead of FastQ/SimpleQue/Whatever".

They key to that second step, is to have your product abstract the queue functionality it needs, into an interface that it uses to talk to the queue - allowing people to swap out queue backends with a simple configuration change.

So, make it simple to get started - and simple to scale up when you decide to.

[ankitnayan]

hmm..got your point. We shall definitely look into other queuing system to be integrated as interface. Trying to understand better, what's a super simple dev setup like (to get the adoption)?

Right now, we can run SigNoz with all components including Kafka and Druid in 4GB memory supporting around 200 events/sec. Though, will need to check whether this micro setup passes a run of a few days.

[dflock]

What you have now isn't too bad, but figure out if you can get it down to one single command. Have a look at what netdata does (https://learn.netdata.cloud/docs/agent/packaging/installer#a...) - this is a single command to install, works really well and is super quick to get started on a single node.

[gen220]

Redis is like this. It runs in-memory by default, but can be trivially configured to write to disk / be persistent between server sessions. Many credit this feature as a catalyst to its adoption.