Languages that rely on the user creating their own language constructs suffer from a balkanization of the language - each team creates their own language, and cannot share code.
I agree it's much easier to make a mess, but I can see doing it the right way being extremely powerful.
One (slightly contrived) example is javascript's different (sub?) languages. JSX, await/async, typescript, pre-processors, etc.
It would be best practice to pull in popular versions of these things as opposed to rolling your own.
However, I also don't think there's much issue with some custom languages as part of a normal codebase when used in moderation.
There's not that much difference between using a traditional API that requires function calls in the right order, maybe some testing to ensure you're using it correctly, etc.
With a "new" language you can put a lot of that as static-typing style compile requirements. Essentially more guard rails to using a library correctly, and an opportunity to make a set of scoped primitives specific to what problem you're trying to solve. Ruby on rails is the best example of this sort of system.
Again though, I know it's a huge pipe dream, and for these ideas to go forward will require a LOT of change in our intuitions and methods of developing software. I'll never stop dreaming though ;)
There's no doubt it is extremely powerful. But the result is everyone invents their own DSL, which is inevitably undocumented and nobody else wants to touch it.
> However, I also don't think there's much issue with some custom languages as part of a normal codebase when used in moderation.
I've seen it enough. Case in point, back in the 80's a friend of mine worked at Microsoft. A manager came to him with a lament that there was a program that compiled to 50K, written entirely in macro assembler. It had a bug in it. He had assigned it to multiple programmers, each of whom failed after trying to fix it for several weeks.
The problem was the author had invented his own language with the macros, left it completely undocumented, and nobody was able to make heads or tails of it.
My friend said, no problem, he'd give it a try. 2 hours later he had fixed it and checked it in.
Astonished, the manager asked him how he'd figured it out. My friend said he didn't. He ran it through the disassembler I'd written https://www.digitalmars.com/ctg/obj2asm.html (which is why he related the story to me) because it turned object code into asm source code. Found the bug, fixed it, and checked in the new source code.
Not only that, I discovered I could not read my own macros a few years after I'd written the code.
I've seen similar terrible examples that make use of expression templates in C++. They were all the rage for a couple of years. Fortunately for C++ programmers, expression templates are so miserably slow to compile that a blessed damper was put on the exploitation of this discovery.
Feel free to disagree with my assessment of this. You'll have plenty of company. But use macros enough and eventually you'll agree with me :-)
While a fair assumption, we're only seeing a small subset of software jobs here: mostly SV-style big tech. And with that a set of specific technologies that have become popular in the space.
A lot of smaller shops use more unique technologies for different fields that I'm sure are using clojure for some things.
Frankly, I'm just guessing, and would love to see some hard data on this stuff.
This is why D does not have a macro preprocessor.