[the_gastropod]

But you could return the 1000 objects (or less? 1000 records sounds like a lot for any UI to show at once) of 1kb size and allow the clients to request specific pages with a request parameter. There may be applications where you need to ship the full 1M records I guess, but that seems like very much an edge case as far as web apps go.

[tauchunfall]

True, you would not return 1000 objects at once to the frontend.

I first thought it's just a backend use-case, where processing 1000 records in a paginated result is common, but the parent mentions "rails", so it sounds like a frontend use-case.

[jupp0r]

It's a backend use case.

[wredue]

1,000 records is absolutely not a lot on modern computers or connections. On a business LAN, this request should take well under a second full latency.

On an average mobile connection, it’s maybe a second or so.

[the_gastropod]

You’re right. It’s not a lot for a machine. The point isn’t the speed capability. It’s why? What UI has 1,000 rows in, e.g., a table all at once (much less 1M)?

[radus]

Many plots contain thousands of data points. Eg: 10 x 100 heatmap which supports sorting by various metadata. This is a common visualization for biological data where your data matrix is samples x proteins, so potentially much larger than 1000 data points.

[jupp0r]

Your assumption that humans consume this data is wrong. It's actually machines that need it.