Hacker News new | ask | show | jobs
by TeMPOraL 2 hours ago
More-less, yes. I was actually thinking about taking a high-resolution rendering of an awning directly facing the camera, and transforming that quad onto the user image - which requires computing the transformation matrix that would right the user image so the building is level and directly facing the camera, and then applying the inverse of that to the quad with your rendering - but I now realize I assumed user photos would mostly be nearly straight images of the building, not at large, odd angles. For general case, you'd need a 3D model (even if approximate), and apply the inverse transform to that, and then render it on top.

This idea rests on the assumption that my understanding of what "awnings" are is correct and matches your project, i.e. additive structures. In that case, your primary problem is adding pixels on top of the user image. Additive modifications are easy to pull off. Inpainting seems like overkill here; it's something that shines when you need to poke holes or replace some of the aspects of the original that is not covered by the part you're adding.

OTOH, it might still be that inpainting is your best bet for operational reasons - additive modification itself may not be a problem, but fixing lighting and shadows might, and current image generations models should handle this in stride.

(I say should because that's my expectation, but I never tested any of the current models on for ability to fix shadows that cover areas similar to the targeted modification, but lay beyond it. It might be that you'll still need a model and a transform estimate just to generate a shadow map as a hint for the model where it needs to act and how.)