Hacker News new | ask | show | jobs
by anonred 1812 days ago
Seems quite disingenuous to claim low latency and variance based on a synthetic benchmark when the real world p95 latency and variance will likely be quite high with a normal distribution of user geos.

And FWIW I requested the same link on my laptop and then phone and my phone took >1s to resolve the redirect.

1 comments

I appreciate the response. I clarified the method that I am using on the comparison page, but did not want to go into too much detail because it was taking more real estate than I wanted it to.

I will extend that text by saying 1 ping is taking the average time of hitting the URL 5 times. The reason for taking 5 readings directly after each other is to filter out any network related issues and noise on the graphs.

When you see a spike in the x axis, it means that was the average latency of hitting that URL 5 times for that interval(every 10 minutes).

I am doing it every 10 minutes because I want to show the variance not only for a single region but especially across regions. If we were to take the p95 or even p99 tail latencies over say 100k clicks, WarpURL will be significantly faster.

I don't think I am being dishonest in my claim?

I don't see how you can get a sub 100ms latency average over 5 hits if your cold starts are > 1s.

Maybe hitting the URL every 10 minutes keeps the URLs you test in your cache.

If users are frequently getting >1s due to cold starts and your stats are systematically excluding them, then the stats aren't very realistic.

I'm guessing you can see how many cold starts there were in AWS Lambda somehow... I can say anecdotally that I got slow responses several times from link.warpurl.com

> I don't see how you can get a sub 100ms latency average over 5 hits if your cold starts are > 1s

I can't and I am not trying to hide it, the CDN will keep a link in its Edge Cache for about 1 day. This is what causes the smaller spikes on the WarpURLs chart, the test link gets evicted from the cache and has to be looked up in the origin. WarpURL does not optimize for the first hit, but for the thousands that follow it near that PoP.

---

> Maybe hitting the URL every 10 minutes keeps the URLs you test in your cache.

No they stay at least 24 hours

---

>If users are frequently getting >1s due to cold starts and your stats are systematically excluding them, then the stats aren't very realistic.

Again, WarpURL does not optimize for the first/single click but for a single URL that gets thousands and thousands of clicks.

---