[dinobones]

I don't understand the "magic" here.

In a traditional approach, you would generate random images, calculate some distance metric, then use some optimization method like simulated annealing to minimize the distance.

I get that the difference between the image representations is being optimzied here, but how is it possible that changing tokens in a program is differentiable?

[revalo]

Changing tokens in a program is not differentiable. For me, the key idea is that you can train a neural model to suggest edits to programs by randomly mutating nodes. And when you run this neural model, you get to make edits that are syntactically correct (i.e., a number will only replace a number etc.) according to a context-free grammar.