Ask HN: Why isn't AI image generation closely linked with graphics code gen?

[amichail]

For example, when making an iOS game, you might want to ask the AI look at a screenshot of your game and to generate images with improved graphics.

You could then pick one of the AI images that you like and then ask the AI to write Swift code to generate scalable graphics like that for your iOS game.

Why is AI image and code generation not currently optimized for this use case?

[heyitsguay]

The two outputs (an image and a string sequence) are learned from entirely different data, even when they're generated with the same model, which is not always the case. There's nothing intrinsic to the learning process that connects image generation and procedural graphics code.

If one wanted to try that, it would likely take the standard LLM finetuning route - scrape a bunch of data pairs at some expense, write some prompts, and have at it.

[robthebrew]

GIGO. Have you asked AI?