Hacker News new | ask | show | jobs
by cheriot 2741 days ago
I'd be curious to see a cost estimate for some traffic level. I wonder if there's a way to put the pixel in s3 and process the access logs more cheaply.
3 comments

I’ve seen folks put their pixel endpoint behind Fastly and process the access log delivered in S3. A Fastly VCL can handle the same transform that this Lambda is doing.
We have people doing exactly this with fly.io, you could also do it with lambda@edge if you're a masochist. Or with Cloud Flare workers if you dislike small startups.
Is fastly free? Why would they use fastly and not s3?
S3 access logs alone are not sufficient to replicate this pipeline. This pixel is stateful (for the anonymous user ID) and S3 access logs don’t include arbitrary headers, in this case the cookie with the user id. Fastly would let you eliminate API gateway, Lambda, and both Kinesis steps.

API Gateway by itself is $3.50/million requests, which is 2-4x more expensive than Fastly at $0.75 - $1.60/million

I've done this a few times and have found it to be an extremely effective way to do simple pixel tracking (for custom emails and the like).
Here is some more data: http://highscalability.com/blog/2018/4/2/how-ipdata-serves-2...

I don't understand what's cubejs doing in this app?

Once data is inside athena, it's matter of querying it right.