3-D Tool Guesses What a Photo Is Missing

A new digital editing tool allows users to turn and flip objects in a photograph, exposing areas not even captured in the original image — turning a chair or flipping a car, for example, to reveal hidden backsides and undersides.  The software relies on 3-D models of everyday objects that are easily available online to “imagine what is not visible,” said Yaser Sheikh, a computer scientist at Carnegie Mellon University and a member of the team that developed it. Using numerical data from the 3-D model, the software tries to guess what the selected object might look like if moved. The software also uses the pixels in the image and the 3-D model to estimate the color, textures and lighting of the object when it is moved. “It infers the 3-D lighting of the whole scene to create somewhat realistic shadows,” Dr. Sheikh said.

He and his colleagues will present their findings this week at theInternational Conference on Computer Graphics and Interactive Techniquesin Vancouver, British Columbia. The editing tool and its source code are available online at no cost. Dr. Sheikh’s doctoral student Natasha Kholgade led the project and will decide whether she wants to commercialize the technology.