[Spidler]

SQLite scales very well for reading data, actually. If your site is single-write:many-read (most are) it works brilliantly.

If you however cause writes when people open pages, look at content. Anything from hit counters to tracking behaviour. Then SQLite suddenly starts to scale pretty badly.

The same goes with how your processing works. Multiple processes that may all at some point cause writes? This is bad. Threads inside a process? That works.