Seeing images spring to life feels like the future of Instagram. Dr Jingtang Liao demonstrated his software that creates depth in 2D images. The next step is to bring this magic into the hands of smartphone users.
Adding depth to pictures. (Images: Jingtang Liao)
Once a picture contains depth information, it can be used to amplify the perception of depth by changing the depth of focus, the lighting, haziness or perspective. But how to move from a flat RGB (red, green, blue values for every pixel) picture to what Dr Jingtang Liao calls an RBGD picture that has an added value for depth in each pixel?
Liao, who graduated last week at the Faculty of Electrical Engineering, Mathematics and Computer Sciences, has developed software that makes it easy for users to annotate depth in images. Darker is closer, and further away is lighter. By choosing a matching grey tone, the user adds scribbles to the picture to add depth information. In the case of Vermeer’s Girl with the Pearl Earring, for example, the shoulder would be almost black, the neck a shade lighter, the face mid-grey, her turban lighter still and the background white. Merging the annotations with the spatial colour distribution, the software then creates a depth map in grey scale.
Darker is closer, and further away is lighter
The software will assume gradual depth transitions, except where there are sharp edges in the picture, in which case the software will assume a sudden jump in depth. Sometimes the user has to identify regions to be ignored, such as shadows and reflections in windows.
Once the RGBD map is complete, the fun can start. Depth can then be used to increase depth perception by changing saturation, haze, focus, or the perspective. Changing the point of view makes objects move more as they come nearer. So when you move the point of view horizontally, vertically or just wiggle it around, the image will spring to life as if by magic. Watch the video to get an impression of the effect.
Another method to add depth to flat images is the use of depth bars. Depth perception can be strongly enhanced by placing parts of the picture in front or behind the bars. But the placement of the bars is critical for the effect. Liao has used depth information to automatically generate depth bars for the optimum effect.
Although depth bars, alias Fake 3D, are already used in commercials and music videos, Liao believes their use will become more widespread, sophisticated, and diverse. Video producers will add depth by using different shapes and having the depth enhancers fade in and out.
Liao has left for China where he will join smartphone producer Huawei. Shortly before his departure, Liao predicted that the next generation smartphones will feature twin cameras to generate images with depth. This RGBD format will be the starting point for personalising one’s photos with a range of 3D effects. Prepare for a time where static images will be a thing of the past.
- Jingtang Liao, Techniques For Depth Acquisition And Enhancement Of Depth Perception, PhD thesis supervisor Professor Elmar Eiseman, December 12, 2017. Click to download as pdf
Do you have a question or comment about this article?
j.w.wassink@tudelft.nl
Comments are closed.