[dev_dull]

My assumption was that the system had finite ram available, whether that's serialized in a database or read directly into the program.

[gwbas1c]

So? Typically "database" implies a product like SQLite or Postgres. These handle many multiples of RAM size.

In-memory databases are a thing, but that's a very specialized use case.

[dfox]

Somewhat surprising fact about most PostgreSQL deployments is that most of the frequently accessed tuples will fit into RAM and it handles this usecase really well (to the extent that people who don’t know any better do not see any issue with LIKE ‘%whatewer%’ over 100M tuple table, it just takes 1.5s, so what...)

[PhearTheCeal]

Pretty much all SQL DBs support pagination, and many libraries that abstract the DB away hide this as an iterator.