I'm consistently surprised with Etsy engineering. (I guess that means I am not updating.) For what appears to my eye to be a rather generic "e-commerce marketplace" (my only visibility is regretsy) they really do a lot of big engineering projects.
I would have expected that a public company in this position would scale back engineering since it doesn't, at first glance, seem to be directly relevant to their business (they're not a tech company). How does a company end up with "good" management that keeps the tech so strong?
I would guess it has something to do with tough lessons learned from their catastrophic early technical debt that brought the site and development to somewhat of a standstill right around 2009 or so.
Many of the acknowledged leaders from a tech architecture standpoint started from the point of their architecture being a real mess. Soundcloud and Spotify come to mind.
I thought you meant hitting "return" on the keyboard, but then I spent 30 seconds trying to go back to HN with a keyboard shortcut and discovered what you meant.
This looks pretty cool and very slick, though it seems very similar to what Graylog[1] offers out of the box, which we've been using in production for some time now.
Correct me if I'm wrong, but wouldn't this allow for a much richer set of alerts since you're able to use the elastic search query syntax? This also allows you to alert on an empty result set which can be pretty handy.
Just got this set up internally and it seems pretty slick, but my chief complaint is the heavy focus on Logstash (which we don't use), instead of Elasticsearch (which is really what's meant). Also, it seems to be missing support for wildcard indices or hourly (non-daily) indices in Elasticsearch.
It looks like Etsy's security team built this to replace functionality they depended on in Splunk that was missing in Elasticsearch. The slides don't make clear why they moved away from Splunk.
Has to be cost. With how much they charge I'm still surprised that there's no startups offering comparable (particularly on-prem) products. May be it's a harder product to make than it seems.
> I'm still surprised that there's no startups offering comparable (particularly on-prem) products
Please consider giving EventQL [0] a try some time! It's completely open-source and self-hostable. Still a new project though, just released this summer and still in beta.
"EventQL is a distributed, analytical database. It allows you to store massive amounts of structured data and explore it using SQL and other programmatic query facilities."
So it's a completely different class of application than splunk or elasticsearch, and one that you have a commercial interest in. Please don't spam HN.
>> So it's a completely different class of application than splunk or elasticsearch
Sure it takes a somewhat different approach (i.e. it requires an explicit schema), but for the use case discussed in this thread it _is_ completely relevant and a comparable open-source/on-premise alternative which parent was asking about.
>> one that you have a commercial interest in
Yes, I'm involved in the EventQL project but I thought that it was obvious from the way I phrased my posting. Usually I always include a disclaimer to prevent misunderstandings but I didn't consider it necessary in this case.
> Configure Searches to periodically run against a variety of data sources. You can define a custom pipeline of Filters to manipulate any generated Alerts and forward them to multiple Targets.
When I read this I thought it was some sort of scraping service or something, a bit like IFTTT where you can hook things together
TL;DR - it's for alerting on ELK, claimed to come with a better query language than both SPL and Elastic QueryDSL (for the use case at least), and a decent admin UI for managing the queries.
I would have expected that a public company in this position would scale back engineering since it doesn't, at first glance, seem to be directly relevant to their business (they're not a tech company). How does a company end up with "good" management that keeps the tech so strong?