Not to be too pedantic, but code is a kind of specification. I think making the blanket statement "Prompt is code" is inaccurate but there does exist a methodology of writing prompts as if they are specifications that can reliably converted to computational actions, and I believe we're heading toward that.
The machine model for natural language doesnt exist - it is too ambiguous to be useful for many applications.
Hence, we limited natural language to create programming languages whose machine model is well defined.
In math, we created formalism to again limit language to a subset that can be reasoned with.