[ticklemyelmo]

Isn't the location that a ray strikes entirely a function of angle of incidence, its frequency, and the particular lens? Why couldn't that function be determined and inverted?

[proudfoot]

Optimally yes. But in practice multiple rays from multiple angles strike the same sensor.

Lens design is about minimizing this.

[Fronzie]

The phase matters as well if some parts are captured out of the focal plane.

Even if reconstruction is possible, it might amplify the noise by a substantial amount.