Hacker News new | ask | show | jobs
by KorematsuFred 2217 days ago
I think you have got it right. I own dozens of static websites from 100 pages to 1M pages. I am the ideal customer for a company like Gatsby.

I had all the content in my mysql and accessible through a REST api. I had python scripts that would pull data from REST and generate .MD files which then Jekyll would convert to static sites. Everything worked fairly well except long build times for the 1M pages site which was expected.

I decided to try porting things to Gatsby because I thought I can get rid of the python script. Then I realized how useless Gatsby suddenly is.

It will take hours to call the REST APIs and create nodes. It would take huge amount of memory too often causing the process to crash even for sites that had just 10,000 nodes. The create pages step was even more horrible. If there is some error like a special char in the url it would just crash the whole process forcing me to build from scratch again which again took lot of time. I had to then try to figure out how to cache the data. Even then, memory crashes were frequent and Netlify free plan would not support the long build times.

I then decided not to use Gatsby for my use case and instead just use Google Cloud Run and PHP for the super large site and continue using Jekyll for all the < 1M pages sites with free Netlify plan.

Now, if you are going to raise lot of money from VC I would like to see you making products that make your customers happy and not engaging in some of engineering porn where you try to solve problems like using "react components" which in itself has no meaning to me.

3 comments

Have you tried testing other generators for sites with 1M pages? How about Hugo?

I'm confused about Gatsby as well to be honest. It has a lot of buzz but Hugo does everything I need. It can download data from URLs too and, worst case, just write a short script that does this before triggering the static site generation.

Part of the appeal of static site generation for me is simplicity. I've already hit enough issues with Jekyll having versioning issues when I return to a site I've let sit for a while. I don't want to have to deal with versions and upgrading issues that'll be involved with using Node, Webpack, React etc.

A million statically generated pages seems to be an interesting use case (also your use of cloud run suggests it's not high traffic). Is this common at all though?
Big documentation sites.
Million pages? What is the site about?