How would not having the source code be present/pre-generated, and thus needing to generate it at runtime, be an example of a decorator having no runtime effects?
OP has to be doing some parsing somewhere, so you just switch to seeking decorators rather than magic comments. It's still before the code reaches the interpreter.
The potential issue I see here is that comments are valid anywhere while decorators may not be, but the parser is hopefully resilient to that. You could see a multi-phase LLM that uses the interpreter to ensure the code runs / works as expected
I’m not objecting to what you say. The latter part of my comment is simply considering cases where you only want to replace a part of the function (the author have some examples on the project page), where a decorator wouldn’t be flexible enough to support. Of course you can also argue those can be easily rewritten to be function scoped, but that is a design tradeoff you make that author did not (or didn’t want to).
in my question, there is no need for the decorator to be handled at runtime, the tool that does the LLM stuff has to parse the Python code to know what to generate. It can just key off of decorators rather than comments, or at least this is my question & hypothesis.
The decorator invokes AI completion only the first time the function is run.
edit: I lost interest before I was able to get arguments to work ¯\_(ツ)_/¯