[orf]

Whilst shallowly funny the sarcasm actually shows a lack of understanding of the problem.

The problem can be described as this: given an arbitrary point in a program, how can you infer what data is required at a future point without executing code between those two points?

I’d love to see you go make a solution to this.

[cnity]

The point is that there is no "generic" solution. The abstraction of the ORM has created the illusion that you don't need to worry about this, the ORM will handle it. N+1 examples like iterating over all the publications from an author arise because the goal is to not have to concern yourself with the fundamentals of data fetching.

[orf]

I’m not sure that position lives up to reality. It’s easy to avoid N+1 queries in Django, there are lots of ways to reduce that to 1 joined query or 2 in the case of a m2m. The issue here isn’t that “orms are bad”, the issue here is that “knowing how data flows through your system is hard”.

You’d have exactly the same issue as you’d have if you’d call “get_author_publications” O(n) times in some nested call stack. Except getting out of it wouldn’t be as easy as calling “select_related(…)” - you’d instead just end up re-inventing an ORM.

[cnity]

> Except getting out of it wouldn’t be as easy as calling “select_related(…)” - you’d instead just end up re-inventing an ORM.

This could be where we differ. I wouldn't propose that at all, I'd propose writing a query more appropriate for whatever part of the code contains the iteration.

[orf]

> I'd propose writing a query more appropriate for whatever part of the code contains the iteration.

This is an identical solution to adding “select_related()” in the right places, and thus we loop back to the start of our thread.

[hbrn]

> The abstraction of the ORM has created the illusion that you don't need to worry about this

It's not an illusion, it's an abstraction that sometimes leaks. But all abstractions are leaky.

There are plenty of projects that never reached a point where N+1 became something they needed to worry about. There are plenty of projects where ORM allowed to delay the need to worry about N+1, thus allowing them focus on product.

And that means that abstraction is working fine. It's not perfect, but it's working.