Going transcen dental in the Electrical Engineering Faculty

“You know that movie Being John Malkovitch where an unemployed puppeteer finds a portal to the mind of John Malkovitch?” Dr Willem-Paul Brinkman asks. “He gets inside Malkovitch’s head, sees everything through his eyes, and even manages to control his movements? Ding Ding and I did something similar.”

Brinkman was the promotor of Ding Ding, who defended his PhD earlier this month. Ding has pursued a PhD degree in the Interactive Intelligence Group (Electrical Engineering, Mathematics & Computer Science Faculty) since 2015. His focus is on new computer-supported interventions for social skills improvement, and  phobia treatment, in virtual reality. That work culminated in a virtual reality setting that is mind-boggling, all made with the purpose of training negotiating skills.

Entering your virtual self



^{Photo from Ding’s experiment. (Photo: Ding Ding)}



Indeed, the subjects in Ding’s experiment entered someone’s head. In virtual reality that is, of course. But instead of entering Malkovitch’s head, they entered their own heads. Or better still, they entered a version of themselves, an avatar, that was more adept at certain skills (negotiating in this case). And, as icing on the cake, they heard their inner voice counsel themselves.

‘An easily accessible addition to the total training offer’

Prior to participating in the experiment, the subjects made recordings of their voices. These recordings were subsequently tweaked to make them sound like their own inner voice. Your inner voice is the voice you hear in your head when you are reading, for example. People experience this sound as different to their speaking voice. The inner voice was then added to VR experiments, a bit like a voice-over in a movie.

To make the experience seem realistic, Ding added a special touch: an eye tracking device on the VR-mask that enables the avatar to register what you are looking at. If you look at an object in a room, for instance, you will hear your inner voice – your thoughts – say something about that object.

Back to the purpose of this transcendental experience: teaching negotiation skills. This purpose fits Ding’s overarching work on self-efficacy, people’s belief in their own abilities. Self-efficacy can increase people’s motivation and willingness to engage in social interaction.

Negotiating is part of everyday life

Though many people may equate negotiations with buying a new house or car, negotiating is in fact part of everyday life. We negotiate with spouses or children about chores or where to go on holiday, and in a professional setting about our workload or deadlines. That makes the ability to negotiate an important social skill. Hence, it is also an interesting setting to research the effectiveness of a social skills training extended with virtual cognition.

Ding’s subjects, about 50 in total, could observe conversations between an employer and an employee. They could see and hear what was being said, but the people’s thoughts were also made explicit. This inner voice, or virtual cognition, was used to introduce knowledge and principles and to reflect on the process, all from the point of view of the employer.

In the first scene, the employer reflected on the upcoming negotiation. The following scenes ranged from an employee always being late to an employee handing in their immediate notice. “The participants could thus experience the theoretical side of negotiations being applied to real-life situations,” Ding explains.

‘Participants could build up confidence’

“Taking part in a social skills training is awkward. By giving people a virtual experience, they can build up enough confidence to take part in real training. So a system like this can be an easily accessible addition to the total training offer out there,” says Brinkman.

The research may have a wider scope. Having difficulty with social interaction can also have a pathological cause. The researchers believe that the inner voice concept may play a role in therapy sessions in the future.

Ding is planning to further develop the technique to help people with autism. And Brinkman wants to continue the work to see if the set-up can be used to help people quit smoking.