Scanning paintings in depth

As a master’s student, Tim Zaman worked in unusual surroundings. In the weeks before the opening of the Rijksmuseum, Zaman scanned some of the Rembrandts in the Honours Gallery (Eregalerij) whilst football star Ruud Gullit and former princes Maxima walked past on their VIP preview tour. Zaman was there to scan some paintings in 3D to obtain a high-resolution dataset. Canon-Océ wanted to feed those data into their newly developed 3D printers to make relief replicas of the masterpieces. And although the result looked stunning, it also made clear that the old masters had some tricks up their sleeves that we cannot yet replicate.

The new Océ relief printer was in fact the impetus for the research project that Professor Jo Geraedts (IDE) contacted art historian and chemistry Professor Joris Dik (3mE) about. Tim Zaman took up the challenge of developing a topographic imaging device and was supervised by robot vision expert Professor Pieter Jonker (EEMCS) and artificial intelligence specialist Dr. Boris Lenseigne (3mE).

“Depth in a plane shows up as small displacements that are different for both eyes,” explains Zaman about the working principle of his scanner. A point, in the middle of your view, which moves towards you, will be moving left for your right eye and right for your left eye.

However, there is ambiguity if the point is not precisely identifiable, for instance because it’s part of a selfcoloured spot. To remove such uncertainties, one can use a dark and light striped pattern to project over the painting. The pattern makes every point identifiable.

Capturing 40 million points in a 10 by 17 centimetre surface took about two minutes. Reconstructing the x,y,z coordinates took about fifteen minutes of calculation. Scanning Rembrandt’s painting The Jewish Bride took 240 captures followed by two weeks of photo-stitching and reconstructing before every point had the right x,y,z and RGB (colour) data. The planar resolution is 50 micrometers, depth resolution a mere 9 microns.

The set-up with two 40 MP cameras and a mini projector on a translating standard was auto-controlled by a smart device that Zaman designed and built. When people first see the 3D prints made by Océ, they are usually impressed. The reproductions make you realise how much the texture adds to a painting. Zaman observes, “When Van Gogh paints flower petals, he’ll only put one or two paint strokes. The highlights and shadows on the paint precisely match that of petals.” Instead of painting certain structures, painters sometimes rebuild them.

That said, put the reproduction next to the original and you’ll appreciate the differences in colour depth and transparency. Paradoxically, improving the technology of replication has taught Zaman respect for the old masters. “I think they had an enormous knowledge of their materials and they knew how to apply that to create unique visual effects.”

Tim Zaman, ‘Development of a Topographic Imaging Device For the Near-Planar Surfaces of Paintings’, 20 February 2013, Supervisors: Prof. Pieter Jonker (EEMCS) and Dr. Boris Lenseigne (3mE).