Silent speech

If a talking mouth is all there is on the screen, the automated lip reader that Dr Alin Chitu has developed even beats experienced human lip readers. However, as soon as the image widens, humans win hands down. Context is perhaps the most important factor for humans in visual speech recognition. You needn’t be especially gifted to understand the request for a ‘Biertje?’ in a crowded, noisy bar.
For his PhD project at the faculty of Electrical Engineering, Mathematics and Computer Sciences, the Romanian-born Chitu compared three different methods for lip reading, setting up ‘corpus’, or database, of twelve hours of video recorded spoken Dutch that contains spoken digits, letters, texts read aloud and bits of spontaneous speech. As for which is the best method, that depends, Chitu says. At high speech rates, the method that continually fits a mouth shape over the camera image seems to perform best. At lower speech rates, the method that only captures the movement of the mouth is the best choice.

So far the system recognizes 92 percent of the spoken digits correctly; 60 to 70 percent of spelled letters; 50 percent of fenced-in talk (in a fixed menu) and about a third of free speech. Currently, the prototype needs about two to three times the utterance duration for processing, which is quite fast considering how many different options the system must weigh and discard.
Chitu’s PhD supervisor, Professor Leon Rothkrantz, thinks the first lip-reading application will be in mobile phones. “It’s a huge market,” he says. Recognition of silent speech would not only benefit rail commuters, but also crisis communication in noisy environments. Ever tried making a call from a helicopter? Yet another application could be in the very labour-intensive process of teaching hearing-impaired children to lip read.

Dr Alin Chitu, Towards Robust Visual Speech Recognition, 2 November 2010, PhD supervisor Professor Leon Rothkrantz.

Naam: Bram Beernink (20, Technische Informatica)
Merk: Favorit
Gekocht voor: Van moeder overgenomen
Opvallend: De fiets is van Tsjechische makelij

Al van ver hoor je Bram Beernink aankomen. Of beter gezegd, hoor je zijn fiets aankomen. “Ach, hij brengt je van A naar B”, zegt Beernink gelaten. Hoewel hij wel graag een andere fiets zou willen. Heel begrijpelijk, gezien de staat van de fiets. De koplamp bungelt aan een draadje, spatborden zijn doorgesleten, de trappers staan op half zeven en de terugtraprem doet het slecht. “Laatst ging het bijna fout op het kruispunt, ik kon nog net op tijd tot stilstand komen.” De gebreken hebben waarschijnlijk alles te te maken met de leeftijd van de fiets. “Hij is van mijn moeder geweest”, zegt Beernink, “dus misschien is ie wel ouder dan ik ben!”