[cnity]

If only there were some way to retrieve information from disparate tables in a single request. Almost like some way to "join" the tables together... you could design a simple declarative language specific for querying in such a way. A "query language" if you will. It would have a simple structure, a structured query language, that enables you to get this data in a performant way without making N requests!

I might go and make this.

[orf]

Whilst shallowly funny the sarcasm actually shows a lack of understanding of the problem.

The problem can be described as this: given an arbitrary point in a program, how can you infer what data is required at a future point without executing code between those two points?

I’d love to see you go make a solution to this.

[cnity]

The point is that there is no "generic" solution. The abstraction of the ORM has created the illusion that you don't need to worry about this, the ORM will handle it. N+1 examples like iterating over all the publications from an author arise because the goal is to not have to concern yourself with the fundamentals of data fetching.

[orf]

I’m not sure that position lives up to reality. It’s easy to avoid N+1 queries in Django, there are lots of ways to reduce that to 1 joined query or 2 in the case of a m2m. The issue here isn’t that “orms are bad”, the issue here is that “knowing how data flows through your system is hard”.

You’d have exactly the same issue as you’d have if you’d call “get_author_publications” O(n) times in some nested call stack. Except getting out of it wouldn’t be as easy as calling “select_related(…)” - you’d instead just end up re-inventing an ORM.

[cnity]

> Except getting out of it wouldn’t be as easy as calling “select_related(…)” - you’d instead just end up re-inventing an ORM.

This could be where we differ. I wouldn't propose that at all, I'd propose writing a query more appropriate for whatever part of the code contains the iteration.

[orf]

> I'd propose writing a query more appropriate for whatever part of the code contains the iteration.

This is an identical solution to adding “select_related()” in the right places, and thus we loop back to the start of our thread.

[hbrn]

> The abstraction of the ORM has created the illusion that you don't need to worry about this

It's not an illusion, it's an abstraction that sometimes leaks. But all abstractions are leaky.

There are plenty of projects that never reached a point where N+1 became something they needed to worry about. There are plenty of projects where ORM allowed to delay the need to worry about N+1, thus allowing them focus on product.

And that means that abstraction is working fine. It's not perfect, but it's working.

[textread]

Would GraphQL be a good starting point for such a language?

[bibanez]

It was sarcasm, he was referring to SQL lol

[boxed]

select_related in Django does this. There's also prefetch_related in Django which is not something that is easily done in standard SQL.

[eurasiantiger]

Yes, just transform it to an SQL statement.