Hi! I work on Vector. For a motivating example, let's say you have an application fronted by nginx. Using Vector would allow you to ingest your nginx logs off disk, parse them, expose status code and response time distributions to prometheus, and store the parsed logs as JSON on S3.
There are obviously plenty of ways to accomplish that same thing today, but we believe Vector is somewhat unique in allowing you to do it with one tool, without touching your application code or nginx config, and with enough performance to handle serious workloads. And Vector is far from done! There's a ton more we're working to add moving forward (thinking about observability data from an ETL and stream processing perspective should give you a rough idea).
Our company uses Splunk. I am not on admin/ops side so possibly missing details. The way I understand is that there is Splunk forwarder running on our app servers. And then there is Splunk server URL from there I get consolidated logs in browser where I can search and run many other statistical function.
So is Vector like Splunk forwarder or more than that?
Vector can act as a Splunk forwarder, but is designed to be much more flexible.
In addition to forwarding to more storage systems (S3, Elasticsearch, syslog, etc), Vector can do things like sampling logs, parsing them, and aggregating them into metrics. Depending on your needs, this makes it easier to reduce your Splunk volume and reduce costs, transition to something like an ELK stack, etc.
We're also working to build up the metrics side of Vector's capabilities. In a way, you can think of Vector as a stream processing system for observability data, capable of feeding into a variety of storage backends.
Thanks for your interest! And please feel free to get in touch if you have any questions or feel there are things we could do to better support your use case: https://vector.dev/community/
There are obviously plenty of ways to accomplish that same thing today, but we believe Vector is somewhat unique in allowing you to do it with one tool, without touching your application code or nginx config, and with enough performance to handle serious workloads. And Vector is far from done! There's a ton more we're working to add moving forward (thinking about observability data from an ETL and stream processing perspective should give you a rough idea).