[breckinloggins]

If you marked shadows and associated them with their source, could you then recover the light source(s) and be able to remove the baked shadows and recast them in real time?

Also, with the shiny objects, could you specify the material properties and have it "back out" the reflection such that the reflection was recomputed as you moved the shape around?

[gohrt]

Yes, there are other projects that do things like insert a synthetic object into a scene, with natural in-context lighting that is inferred from the light gradients on other objects in scene.

[op12op12]

Yes, here's a cool demo video from 2011 SIGGRAPH Asia...can't even imagine how much more things have progressed since then:

http://vimeo.com/28962540