Blender has a great Python API[0] that allows you to do almost everything that you could do in the editor with code.
You do have to run your code from inside the Blender app though.
I'm unsure of why you say "almost" everything when the UI code is simply calling the python API to do it's function. You could do anything and _more_ in the python API, as it's what the UI uses but you can use all the parameters via the code while the UI doesn't expose everything.
Unless something changed recently regarding this.
Also, regarding executing the code from outside, you can run blender headlessly, running the code with it when running blender via terminal. So you could use it as a windowless client if you so wish.
Blender is a truly fantastic tool, but some of the scripting is a bit hacky, IMO.
I recently built some pretty complex Blender automation[1], and my experience was that while technically nearly everything that can be done in the UI can be automated, there are problems that fell into two main buckets for me:
1 - There are basically two parts to the Python API - bpy.ops, and everything else. If something is in the "everything else" category, it works pretty well, but not everything can be done through that part of the API. bpy.ops is (AFAICT) the equivalent of using Selenium to automate clicking on web pages instead of actually interacting with the underlying API, and it feels super fragile.
2 - A lot of the API is not documented well, and I frequently ran into situations where there was no information on what I wanted to do, or the information was outdated. The impression I get is that most of the core Blender devs and users are not using the scripting features, or not using them extensively, because answers on StackOverflow and similar to things like "how can I change property X of every material's shader in my project, oh, and by the way, I have 1000 materials in my project" are usually "I guess you'll just have to click through each of those and set it" even though the API supports it if one digs far enough. A lot of the extensions I thought I could use as reference for things like arranging shader nodes in a useful way have been broken ever since 2.8, but there's no indication in the UI of this.
The pane in the scripting view that continuously shows the script equivalent of actions taken in the UI is a big help, and again, Blender is a really, really fantastic tool. But I can definitely understand some frustration with the scripting feature.
Unless something changed recently regarding this.
Also, regarding executing the code from outside, you can run blender headlessly, running the code with it when running blender via terminal. So you could use it as a windowless client if you so wish.