[benschwarz]

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com. Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.).

Last year I knuckled down to get into the position where I could work on it full time. I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.)

I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!

[mikerg87]

I have been using http://www.webpagetest.org/ for similar investigations. csn you tell me what you see as advantages of your tool over webpagetest.org ?

Btw congrats. This looks very well done.

[benschwarz]

Web page test is a succhhh a great tool, and its really great for one off tests. Calibre will automatically hit your site daily (or, when you ping the snapshot api), and you'll have a full history of what happened and when.

With Calibre the aim is to make it a part of your toolchain (read: integrations) — so it should be visible to your entire team, or build system, or slack channel.

[palerdot]

Nice product. Can you share your tech stack? Do you sub contract some of the development like frontend or you do everything yourself?

[benschwarz]

Sure! Here goes:

The App itself is boring old rails, with postgres (get me started on how much I like postgres and I will literally bore you to tears.) The original views were all haml and sass delivered via the asset pipeline. I sprinkled in angular here or there as the application grew.

Today, I'm slowly tearing away the angular. I've moved to webpack for JavaScript, and React for all new UIs. I've been really enjoying both the speed of development, and how solid everything feels.

Over time, I'm writing less and less ActiveRecord, and more and more sequel… still learning, but getting better.

The app is run on Heroku, and its been A++ would recommend to everyone.

The Calibre agent is node and websockets talking to Chrome®, theres other bits of ruby, and even some golang now.

I use terraform and ansible (although, the ansible may as well have been bash scripts) through a buildkite (https://buildkite.com — the best fucking CI service on the planet) pipeline. AMIs are built, shipped to different regions, then based on some config are rolled out.

Papertrail for logs - omg, logging! Essential. I run thousands of tests per day (I don't know this number off the top of my head, but its a lot)… and the nature of the internet is THERE IS ALWAYS FAILURE. Having logging at my fingertips has always made my job building Calibre easier. I think I pay $20 per month right now, and its the most important thing I pay for.

--

Ok. Whew. Could probably keep going, but I'm not sure how much you wanna know. Happy to answer specific things though!

I do everything. No contractors. Although @fox has been recently helping me with my writing (her native language is Polish, and she writes waayyy better than me - and most others).

[pspeter3]

Is it only clean room metrics or can you also collect metrics from actual users?

[benschwarz]

Atm, the focus is clean room, but real time/user metrics are on the horizon—especially when the company is more than just me!