[jerf]

Abstractions work by restricting the domain of what you can do, then building on those restrictions. For example, raw hardware can jump anywhere, but structured programming constrains you to jump only to certain locations in order to implement if, for, functions, etc. It is precisely those restrictions that bring the benefits of structured programming; if you still frequently dipped into jumping around directly structured programming would fail to provide the guarantees it is supposed to provide. CRUD frameworks provide their power by restricting you to CRUD operations, then building on that. Immutable data is accomplished by forbidding you from updating values even though the hardware will happily do it. And so on.

Escape hatches under the abstractions are generally there precisely to break the abstractions, and break them they do.

Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction. There's no way around it. Careful thought can make using an escape hatch less pain than it might otherwise be (such as the ORM that makes it virtually impossible to use SQL by successfully hiding everything about the SQL tables from you so you're basically typing table and column names by dead reckoning), but that's all that can be done.

One thing to do about this is that just as in the past few years the programming community has started to grapple with the fact that libraries aren't free but come with a certain cost that really adds up once you're pulling in a few thousand libraries for a framework's "hello world", abstractions that look really useful but whose restrictions don't match your needs need to be looked at a lot more closely.

I had something like that happen to me just this week. I needed a simple byte ring buffer. I looked in my language's repos for an existing one. I found them. But they were all super complicated, offering tons of features I didn't need, like being a writethrough buffer (which involved taking restrictions I didn't want), or where the simple task of trying to understand the API was quite literally on par with implementing one myself. So I just wrote the simple thing. (Aiding this decision is that broadly speaking if this buffer does fail or have a bug it's not terribly consequential, in my situation it's only for logging output and only effectively at a very high DEBUG level.) It wasn't worth the restrictions to build up stuff I didn't even want.

[cloogshicer]

> It is precisely those restrictions that bring the benefits [...]

Wouldn't it be possible to say "ok, I'll take those restrictions as long as they benefit me, but once I notice that they no longer do, I'll break them and drop down to the lower layer. But only for those parts that actually require it"?

> Abstractions necessarily involve being irreversible, or, to forestall a tedious discussion of the definition of "irreversible", necessarily involve making it an uphill journey to violate and go under the abstraction.

Why? Not being snarky, I'm genuinely trying to understand this better.

[jfoutz]

I added 2 numbers and came up with 5. What numbers did I add?

You can’t know, because the abstraction (add) destroys information. A “good “ abstraction destroys information that doesn’t matter, or maybe matter in a given context.

You can hang on to all of that extra detail, but it seems like that extra detail slows down drawing inferences.

When I claimed adding resulted in 5, you probably didn’t care if it was 5 apples or 5 skyscrapers. The addition results of 5 are the important part.

Kinda hand waving, but what is included and what is left out is the heart of abstraction imho. And when it’s left out, you can’t get it back.

[ulbu]

You are right.

How Aristotle illustrated abstraction – by extracting the definition of triangle. Reversible abstraction would mean the possibility of taking a triangle in the Cartesian space and reconstructing the actual object that includes an instance of this triangle. We’d have to be able to reconstruct its material properties.

It is precisely this impossibility that is essential to abstraction. It involves the removal of properties of a particular to arrive at the abstract – the common, the universal.

Now, it is possible to arrive at a poor, incomplete abstraction. Imagine if we dealt with red triangles and green triangles, as opposed to just triangles. If we wished to operate on the its underlying triangle, we would have to remove the colour at each operation. It would be an unused variable. We don’t want that – so we remove the property of colour from our triangle operations entirely. And thatbis the only way to deal with triangles.

Reversibility is simply a different property that can’t be attached to abstraction is if we wish precision at all.

[s_m_t]

I think there is something to the nature of your addition example that is counter to your point. Addition being an "abstraction" doesn't make it destroy information it's just an algebraic property of addition. If you instead took multiplication of prime numbers as your "abstraction", no less abstract than addition, then every product would have a unique factorization in the primes. Whether either of these operations makes sense for your problem and thus whether their limitations apply doesn't have anything to do with what "abstraction" you choose. They are either isomorphic to some aspect of your problem or they aren't.

[plaguuuuuu]

I think the parent's point is that it doesn't need to be like that.

You might be an abstracted adding thingy that adds two numbers to get some other number. But you could also implement a visitor pattern that hooks into the guts of your implementation, allowing me, for example, to inject a logger to record what those numbers were.

Likewise, some ORMs have some abilities to bypass the query engine to hand-write a query, or change the way data types are serialized/deserialized - the key here is basically providing an API to peek under the hood when the user needs to do some stuff at a lower level than the abstraction dictates

[drewm1980]

There actually is such a thing as reversible computing(?) that keeps around enough intermediate values to run the code backwards to the inputs. Not sure what it is actually good for. Not to be confused with time travel debuggers, though they are similar in spirit.

[jerf]

Yes, it would be OK to do that. One of the worst things abstractions can do is seal you away from the lower level even though you still need to get there. Another example I recently used was a CRUD framework that 100% took over the routing of URLs and had no callouts whatsoever for a non-CRUD-framework page. A CRUD framework may do many wonderful things for my CRUD pages but I may still need a URL to go somewhere else.

"Why? Not being snarky, I'm genuinely trying to understand this better."

It's almost a definitional issue, honestly. If you aren't doing some sort of limiting and building on limitations, you don't have an abstraction, you just have a library. A library is just a whole bunch of programming you could have done yourself, but doesn't impose any further restrictions on you. A classic example would be an image decoding library. They generally do not impose anything on you. They're just huge bits of code that turns images into pixels. You justifiably pull one in because why would you want to write that code again when battle-tested code already exists, but it doesn't impose anything on your code.

Now, when you start getting an "abstraction" that tries to make it so you can operate on multiple types of images at once, you're going to have some restrictions. Your abstraction is going to make some choices about the color spaces you can operate on. Your abstraction is going to make choices about what features they expose from the underlying graphics formats. For example, consider how a generalized "operate on all images" library will handle animated formats. It's a very sensible answer to say it won't operate on them at all. Or perhaps it offers animation support and errors out if you choose a format that doesn't support them. Can it export SVGs? Can it export SVGs with any sort of embedded Javascript? Does it do anything sensible with imported SVGs with embedded Javascript? What does it do with vector versus pixel based images?

You may say "ah, but jerf, I can easily provide a library that simply allows all those things no matter what... or, well, at least I can hypothesize such a library, providing it would actually be a pain... but there's no reason it couldn't exist". Exist, yes, but what you'd find is that if it let you do everything to every image format that it actually wouldn't do much! You can provide a library that takes a file, figures out what kind of image it is, and hands back a specific instance of a *JPEG or *GIF or *TIFF or *SVG, but if your supposed abstraction lets you do anything with anything, if you sit down and work out what it actually means for my JPEG support to give you access to the DCT tables directly and the GIF library to support the exact GIF animations and SVGs to offer direct manipulation of the SVG DOM and PNGs to offer direct access to all the frame elements in a PNG is that you don't have an "abstraction" anymore! Your "abstraction" is just a whole bunch of libraries bundled together with a thin abstraction around loading a file, and programmers using it this way get no ability to work across file types because they all have a completely different API.

And then you say "But I could offer a subset", which is 100% true, and that would be an abstraction. But then that abstraction would offer no access to PNG frames or JPG DCT tables.

And that's fine. There's no law whatsoever that a single "thing" must either be an abstraction or a library. It would be perfectly sensible to offer a total image solution that had a series of abstractions of varying level of restrictions and varying corresponding levels of power, and also include a full library for deep manipulation of every individual type. It's fine to offer a generalized "Image" type that has a ".ConcreteImage()" method that returns a concrete image that can be used with the libraries and "penetrate the abstraction". My point here is about the fact that if you want more "power", to operate on multiple image types with the same code, to take advantage of simply committing to a page-based interface to the Web and thereby being able to take advantage of the resulting simplifications it offers, to be able to plug arbitrary compression algorithms on to an IO stream, you always have to build these things on a reduction of power of some sort because you can't get any "abstractive" power if you're trying to offer everything to everybody. It's a fundamental fact of abstractions, it's precisely where they get their power from in the first place, and there is no option where you get those benefits without paying the price.

[jerf]

Oh, forgot to mention: Trying to offer everything to everybody has its own antipattern name, the Inner Platform effect. https://en.wikipedia.org/wiki/Inner-platform_effect

One way of viewing an Inner Platform is when you have an abstraction that is so "powerful" in the process of doing everything for everybody that it fails to offer any of the advantages of imposing the restrictions, but it still manages to deliver the disadvantages of abstractions! (Alternatively it can come from having some layer reduced power in the wrong way, and then having a layer above it trying to recover the original power. Either way these are always, always disasters.)

[cloogshicer]

Thank you very much for this detailed explanation! I'll have to let it sink in a bit.