[throwaway675309]

I would argue that this kind of problem is going to become less of an issue overtime, since they're going to have to also solve the issue of suggesting code samples from deprecated API versions - it's likely that eventually they'll figure out a similar way to promote more secure types of code in the suggestions based on Stack overflow or other types of ranking systems.

[visarga]

Yes, the will surely improve a lot and also train users to write better prompts and comments. With millions of users accepting suggestions, then fixing them, they get tons of free labeling. If they monitor the execution errors they got another useful signal. If they use an execution environment they could use reinforcement learning, like AlphaGo, to generate more training data.