I don't think it's a dumb question. And I think you're right: the closer you get to the way the CPU you're targeting likes its memory optimally managed, you're basically heading towards assembly.
I have notions that there can be something 'between' assembly and, for instance, C, but that's a daydream of mine without any details worked out yet.
I’m not sure you’re actually heading down the path of assembly in terms of the language itself, but you’re definitely moving towards the architectural lock-in of assembly.
But maybe thats fine if the language can express architectural-specific decisions (like memory layout) and abstract it away in a more sane fashion than ifdefsc and the rest of the codebase can maintain the pretense of portability.
Low-level memory work, if you care about things like data locations and exact layout of memory, is possible in practice with C. There are cases where you can run into aliasing problems, but controlling the actual memory layout is no problem at all. (There are some limitations.)
I have notions that there can be something 'between' assembly and, for instance, C, but that's a daydream of mine without any details worked out yet.