[nercury]

Pardon me for the tangent (just a general comment not directed to OP).

What I have learned over the years is that the only way to properly use ORM is as a fancy query tool. Build the query, fetch/update data, MOVE THE DATA to separate business objects. Don't leave ORM entities shared across the sea of objects!

Phew, thanks, I got that off my chest.

[arrowsmith]

I wouldn't have believed you until I moved from ActiveRecord (Rails's ORM) to Ecto (Elixir/Phoenix's data mapping library which is decidedly not an ORM.) It's a million times better and I'm never going back.

[atonse]

Ecto is hands down my favorite part of the elixir ecosystem.

It’s so elegant and the Lego blocks (query, schema, change set, repo) can be mixed and matched in different ways.

I’ve even used schemas and change sets to validate API requests and trivially provide very nice, clean, and specific errors, while getting perfectly typed structs when things are validated.

[cultofmetatron]

same, I wish more libraries would go the ecto design route. my ecto queries map pretty close to 1:1 with the sql counterpart. no guessing what the output is going to look like. I spend my time debugging the query and not trying to get the orm to output he query I want.

[freedomben]

Yes, same experience here. I felt (and still feel) that ActiveRecord is one of if not the best ORMs out there, but it was always a source of debugging and performance optimizations/pain, and the trick was basically taking hand-written SQL and trying to get ActiveRecord to generate that. After briefly moving to node.js full time I actually got very anti-ORM, although query building libraries all sucked too which left me unsure of what the best approach is.

Then I learned Ecto/Phoenix and this is truly the best way. Ecto is so close and translateable to raw SQL that there's little to no friction added by it, but it handles all the stuff you don't want to have to do by hand (like query building, parameterization, etc). Ecto is a real breath of fresh air and I find myself writing quick scripts that hit a database in Elixir just so I can use Ecto! I also love how easy Ecto makes it to model database tables that were originally defined by another language/framework or even by hand. Trying to do that with ActiveRecord or another ORM is usually a recipe for extreme pain, but with Ecto it's so easy.

[arrowsmith]

Yeah, I hear some people say that they find Ecto.Query confusing, and I think it's because they never learned SQL properly. That's understandable because it's possible to use something like ActiveRecord for years without ever learning to write even a simple SQL query. But if you have a good grasp of SQL then Ecto.Query is trivial to learn - it's basically just SQL in Elixir syntax.

[cultofmetatron]

> it's basically just SQL in Elixir syntax.

its sql in elixir syntax with a bunch of QOL improvements.

for one thing, I can seperate my subqueries into separate variables

``` sub_q = from(l in Like) |> where([l], l.user_id == ^user_id) |> select([l], %{ user_id: l.user_id, likes_count: count(l.id) }) |> group_by([l], l.user_id)

main_query = from(u in User) |> join(:left, [u], likes_count in ^subquery(sub_q), on: likes_count.user_id == u.id, as: :likes_count) |> select([u, likes_count: l], %{ name: u.name, likes: l.likes_count, }) |> where([u], u.id == ^user_id)

user = main_query |> Repo.one()

```

Being able to think directly in sql lets you perform optimal queries once you understand sql. and imho, this much cleaner than tha equivalen sql to write. it also takes care of input sanatization and bindings.

[noduerme]

adding an off the shelf ORM layer creates so much more opacity and tech debt than writing queries I don't understand why anyone would willingly put one into their stack. Sure, they're neat although I don't even know if they save time. There's something very satisfying about well-crafted queries. And is it ever really well crafted if you can't tweak them to improve their their execution plan? I've never had a client or boss who asked to use an ORM framework. I suspect it's something people think looks cool - treating SQL as OOP - until they run into a problem it can't solve.

[edit] for instance, I have a case where I use a few dozen custom queries on timers to trawl through massive live data and reduce it into a separate analytics DB. Using everything from window functions to cron timers to janky PHP code that just slams results from separate DBs together to provide relatively accurate real-time results. At the end from that drastically reduced set in the analytics DB... sure, I'm happy to let the client summarize whatever they want with Metabase. But those things just couldn't be done with an ORM, and why would I want to?

[nercury]

Yes, I would not put it just anywhere. But I have few rules about ORMs:

- Proper DB design first. You should be able to remove the ORM and DB should still function as intended. This means application-side cascade operations or application-side inheritance is banned.

- No entities with magical collections pointing to each other. In other words, no n to n relations handled by ORM layer. Create in-between table, for gods sake. Otherwise it becomes incredibly confusing and barely maintainable.

- Prefer fetching data in a way that does not populate collections. In other words, fetch the most fine-grained entity and join related data. Best if you craft special record entities to fetch data into (easy with EF or Doctrine).

- Most ORMs allow you to inspect what kind of queries you create. Use it as query building tool. Inspect queries often, don't do insane join chains and other silly stuff.

I would use ORM in one kind of app: where I would work with data that shares records that might need to be either inserted or updated, and there is several nesting levels of this kind of fun. You know, you need to either insert or update entity, if it exists, you should update, and then assign related entities to it, if it does not, then you should insert, and assign related entities to the newly created id. The ORM can easily deal with that, and on top of that it can do efficient batched queries, which would be really annoying and error-prone to hand-craft.

If the app does not require this kind of database with these kind of relations, I would not use ORM.

[thaumasiotes]

> No entities with magical collections pointing to each other. In other words, no n to n relations handled by ORM layer. Create in-between table, for gods sake. Otherwise it becomes incredibly confusing and barely maintainable.

So, I have a database that looks like this. My method was to lay out the database myself, by hand, and then use EF's facility to generate EF code from an existing database. The bridge table was recognized as being nothing but the embodiment of a many-to-many relation and the EF code autogenerated the collections you don't like.

Is this a problem? If you do things the other way around, the ORM creates the same table and it's still there in the database. It isn't possible not to create the bridge table. Why is that case different?

[nercury]

This is more of a preference for bridge to be visible in application. Also the bridge may seem simple at first, but it also may gain associated data, like created_at, order, etc.

[LaGrange]

> adding an off the shelf ORM layer creates so much more opacity and tech debt than writing queries I don't understand why anyone would willingly put one into their stack.

Simple: because if I don't, I'm going to spend the rest of my career explaining why I didn't to people extremely skeptical of that decision. Meanwhile even people like me tend to just shrug and quietly go "oh, an ORM? Well, that's the price of doing the job."

Also, ORMs are an endless source of well-paid jobs for people who actually learned relational algebra at some point in their lives, and that's not a compliment to ORMs.

[lozenge]

ORM is not for writing analytics queries. It's for your CRUD operations. Something like Django Admin would be impossible without an ORM. You create tables for your business logic and customer support or whoever can just browse and populate them.

[couchand]

Wouldn't standard ANSI SQL's information_schema be sufficient to build such an interface? I'm struggling to see how an ORM is necessary.

[DangitBobby]

I consider an ORM to be any SQL generating API, without which it would indeed be impossible to have a generic Admin class to make Admin views in Django.

[randomdata]

Funny how ORM no longer means Object-Relational Mapping.

[DangitBobby]

What should I call a program that generates SQL, executes it, and stores the result in a tuple, object, or whatever data structure in the programming language that I'm using? Does it magically stop being an ORM the second I use a tuple instead of a class instance, or is it now an ORM plus another nameless type of program? Are tuples also objects?

[DanielHB]

Most queries are pretty trivial, ORMs are great for 90% of queries. As long as you don't try to bend the ORM query system to do very complicated queries it is fine. Most (all?) ORMs allow raw queries as well so you can mix both.

On top of that most ORMs have migrations, connection management, transaction management, schema management and type-generation built-in.

Some ORMs have inherently bad design choices though, like lazy loading or implicit transaction sharing between different parts of the code. Most modern ORMs don't really have that stuff anymore.

[kaba0]

How do you map rows to objects? How do you insert into/update rows in your databases? These are the basic problems ORMs solve splendidly. They are for OLTP workloads, and have deliberate escape hatches to SQL (or some abstraction over it, like JPQL in java-land).

I just fail to see what else would you do, besides implementing a bug-ridden, half-ORM yourself.

[blksv]

Rows are tuples, not objects, and treated as such throughout the code. Only the needed data is selected in the form most appropriate to the task at hand, constructed in a hand-written sql query, maybe even taylored to the DB/task specifics. Inserts/updates are also specific to the task, appropriately grouped, and also performed using plain sql. Data pipelines are directly visible in the code, all DB accesses are explicit.

[couchand]

This. The right way to structure database access is a result type per tuple, not an object type per table.

[kaba0]

ORMs don’t mandate mapping the whole table either, you are free to create multiple entities per table/view.

[DangitBobby]

Maybe we need to use a different acronym than ORM, because to me the thing we can all agree we need is code that emits SQL. If you can't agree that projects need generated SQL because SQL is dog water for composition, then we can't really agree on anything.

[blksv]

Probably so: I can't agree with that particular inference.

1. Very often we need generated SQL because writing SQL for primitive CRUD operations is hell tedious and error-prone (as well as writing UI forms connected to these CRUD endpoints, so I prefer to generate them too).

2. Structured Query Language being very poorly structured is indeed a huge resource drain when developing and maintaining complex queries. PRQL and the like try to address this, but that's an entirely different level of abstraction.

3. Unfortunately, when efficiency matters we have to resort to writing hand-optimized SQL. And this usually happens exactly when we terribly need a well-composing query language.

[PeterisP]

I'd argue that "code that emits SQL" is never an inherent need but a possible development time-saver - we need code that emits SQL in those cases (and only those cases) where it saves a meaningful amount of development time compared to just writing the SQL.

[sgarland]

Every RDBMS has multiple connector libraries that solve this for you, without requiring the overhead of a full ORM.

[DangitBobby]

If the connector library solves this problem then the connector library is an ORM.

[sumtechguy]

That is exactly where ORMs help. The problem is all of the other stuff with it. When most people just need a simple mapper. Not something to build their SQL statements for them (which seems to be why most people pick it).

But that comes to the second problem. Most devs I meet seem to be deathly allergic to SQL. :)

One project I had a dev come to me asking me to look at a bug in the thing. Having never seen that particular ORM before I was able to diagnose what was wrong. Because MS ORMs have the same issues over and over (going back to the 90s). You better read those docs! Because whatever they did in this stack will be in their next one when they abandon it in place 3 years from now.

[to11mtm]

> These are the basic problems ORMs solve splendidly.

Depends on the ORM.

I have noticed that typically, 'unit of work' type ORMs (EFCore and Hibernate/NHibernate as examples) prevent being 'true to the ORM' but 'efficient'.

i.e. Hibernate and EFCore (pre 7 or 8.0ish) cannot do a 'single pass update'. You have to first pull the entities in, and it does a per-entity-id update statement.

> I just fail to see what else would you do, besides implementing a bug-ridden, half-ORM yourself.

Eh, you can do 'basic' active-record style builders on top of dapper as an afternoon kata, if you keep feature set simple, shouldn't have bugs.

That said, I prefer micro-ORMs that at most provide a DSL for the SQL layer. less surprises and more concise code.

[bad_username]

For me the biggest reason is automated database initialization and migration. After defining or updating the ORM model, I don't have to worry about manually CREATing and ALTERing tables as the model evolves.

This is compatible with the OC suggestion of using ORMs as a "fancy query builder" and nothing more, which I strongly support.

[devjab]

You always have to worry about your model changes if you run at any sort of scale. Some ORMs will get it right most of the time, but the few times they don’t will really bite you in the ass down the line. Especially with the more “magical” ORMs like EF where you might not necessarily know how it build your tables unless you specifically designed them yourself.

This is where migrations also become sort of annoying. Because if you use them. Then it is harder to fix the mistakes since you can’t just change your DB without using the ORM or you’ll typically break your migration stream or at least run into a lot of troubles with it.

And what is the plus side of having a code-first DB really? You can fairly easily store those “alter table” changes as you go along and have full availability of history in a very readable way that anyone, including people not using C#, Java, Python.

Which is the other issue with ORMs. If you have multiple consumers of your data. Then an ORM most likely won’t consider that as it alters your “models”.

For a lot of projects this is a non-issue, especially at first. Then 10 years down the line, it becomes a full blown nightmare and you eventually stop using the ORM. After spending a lot of resources cleaning up your technical debt.

[DangitBobby]

> And what is the plus side of having a code-first DB really? You can fairly easily store those “alter table” changes as you go along and have full availability of history in a very readable way that anyone, including people not using C#, Java, Python.

The benefits should be obvious if you've used ORMs. They are an object that represents your database data in code rather than in a table where you can't touch it. If you have code that brings data from a database into code, congratulations, you've implemented part of an ORM. Having the data model defined "in code" treats the code as first-class instead of the SQL, which makes sense from an ergonomics perspective, you will spend much more time with the code objects than you will the SQL schemas. Either way, you will have two versions: a SQL version and a code version. You might as well get both from writing one.

If you can read alter table in SQL, you can probably read migrations.AddField in Python, and whatever the equivalent is in the other languages. I still am waiting with bated breath for the problems with much maligned (by some) ORMs to arrive.

[devjab]

The only area of development where ORMs haven’t been the cause for at least some trouble in my career has been with relatively small and completely decoupled services. Even here I’ve had to replace countless ORMs with more efficient approaches as the service eventually needed to be build with C/C++. That being said, I don’t think any of these should have been build without the ORM. The rewrite would have been almost as much of a hassle if there hadn’t been an ORM after all.

I’m not really against ORMs as such. I’m not a fan of code-first databases for anything serious, but as far as CRUD operations goes I don’t see why you wouldn’t use an ORM until it fails you, which it won’t in most cases, and in those cases where it does… well similar to what I said earlier you just wouldn’t have build it to scale from the beginning anyway, and if you had and it turned out it didn’t need to scale then you probably wasted a lot of developer resources to do so.

[noduerme]

I'm not sure if you're talking about creating and altering model tables or if you mean ORMs provide safety in case underlying tables are modified. I'd argue that well-built queries should be resistant to alteration of the underlying tables, and that views and functions and stored procedures already exist to both red flag breaking changes and also to combine and reduce whatever you need without relying on third party code in another language layer to do the lifting.

[blksv]

Doesn't it also mean that any non-trivial migration (e.g. which requires data transformation or which needs to be structured to minimize locking) has to be defined elsewhere, thus leaving you with two different sources for migrations, plus some (ad-hoc) means to coordinate the two?

(I would say that it is conceptually perverse for a client of a system to have authority over it. Specifically, for a database client to define its schema.)

[marcus_holmes]

Agree completely, as does most of the Go community :) Newbie gophers are regularly told to learn some SQL and stop trying to rebuild ActiveRecord in Go ;)

But in .Net, EF is still the most common way of accessing data (I have heard, because I stopped using it over a decade ago).

[lomase]

EF is the common way of saving data.

[rafaelmn]

That doesn't really help you with EF because there's plenty of stuff shared at context level. So depending on the order of queries in the context the same query can return different data.

I hate EF and everything it stands for. :)

[nercury]

Yeah, in a web app, one context per request. In desktop app... I have never used EF there.

[specialist]

> use ORM is as a fancy query tool

As an alternative to query-by-example (QBE)?

https://en.wikipedia.org/wiki/Query_by_Example