Whether it's acceptable or not to submit AI code, it is clearly unacceptable to submit code that you don't even understand. If that's all an employee is capable of, why on earth would the employer pay them software engineer salary versus hiring someone to do the exact same for minimum wage?
The problem is that "AI" is likely whitewashing the copyright from proprietary code.
I asked one of the "AI" assistants to do a very specific algorithmic problem for me and it did. And included unit tests which just so happened to hit all the exact edge cases that you would need to test for with the algorithm.
The "AI assistant" very clearly regurgitated the code of somebody. I, however, couldn't find a particular example of that code no matter how hard I searched. It is extremely likely that the regurgitated code was not open source.
Who is liable if I incorporate that code into my product?
There's companies that scan code to see if it matches known open source code or not. However they probably just scan github so they won't even have a lot of the big projects.
This seems like you don't believe that AI can produce correct new work, but it absolutely can.
I've no idea whether in this case it directly copied someone else's work, but I don't think that it writing good unit tests is evidence that it did - that's it doing what it was built to do. And you searching and failing to find a source is weak evidence that it did not.
There is no way on this planet that an LLM "created" the exact unit tests needed to catch all the edge cases--it would even take a human quite a bit of thought to catch them all.
If you change the programming language, the unit tests disappear and the "generated" code loses the nice abstractions. It's clearly regurgitating the Python code and "generating" the code for other languages.