| About 7 years ago, when I was trying to monitor .Net apps, there weren't that many alternatives available. The Elasticsearch-Logstash-Kibana stack had just gotten off the ground and I needed a way to send it structured logs and a large number of machines. Logary.tech was born to solve that problem. Since then Logary has expanded with excellent support for sending both metrics and tracing data to a large number of targets. In production, I use this setup; client browser -> Logs & metrics to Logary Rutta HTTP ingestion endpoint via Logary JS nginx-ingress -> Traces to Jaeger Agent via opentracing C++ client
nginx-ingress -> Metrics to InfluxDB
nginx-ingress <- Metrics via Prometheus scrape annotation Our NextJS site and GraphQL server:
site -> Traces to Jaeger Agent via opentracing
site -> stdout logs via Logary JS (also get added as Logs in the Span of OpenTracing)
site <- Metrics via Prometheus scrape annotation and prom-client api -> Traces to Jaeger Agent via Logary's F# API
api -> Metrics to InfluxDB via Logary's F# API
api <- Metrics via Prometheus scrape annotation and Logary.Prometheus
api -> Events to Mixpanel via Logary.Targets.Mixpanel
api -> Logs to Stackdriver via Logary.Targets.Stackdriver (hosted on GCP) Also, Kubernetes ships logs via FluentD to Stackdriver in GCP, but they are not structured, and the remaining infrastructural services also send traces to Jaeger if they can. Logary Rutta is a stand-alone log router, written in Hopac + F# (like Concurrent ML), and used by some of the largest Swedish software companies for thousands of logs and metrics per second. It's capable of shipping to a large number of targets https://github.com/logary/logary/tree/master/src/targets Since it talks HTTP and UDP with a number of encodings (JSON, plain, binary), it's easy to plug into an existing infrastructure and existing log shippers. It can also connect point-to-point to itself with a high-perf binary encoding. Because you can send any JSON into it, it's very easy to get started with together with mobile apps. Logary for JS currently has support for user logs, and I'm currently testing rudimentary metrics and browser info. Logary for .Net supports the OpenTelemetry spec, structured logging and metrics. Of course you can pick any toolchain you want, but I've had great success (and great fun!) writing and using the above. You can see I don't keep logs on disk; it causes them to fill up; if your network is down, your service is down, and then you know it's the network anyway. Once in Logary, you can choose where you send them. I've done an analytics/ETL pipeline based on Logary with its Stackdriver+BigQuery+GooglePubSub targets and with Flink, with great success as well. Logary is free to use for non-profit and then I have a pricing calculator on the home page, for when you start selling the software you build. Pricing aside, how Logary is structured and how I've used it might give you some hints on how to do it yourself. |