The University of California – San Diego researchers have devised a pair of ‘4-D goggles’ which enable wearers to be physically ‘touched’ by a movie as they see a looming object on the screen, such as an approaching spacecraft.
The googles were developed out of a study conducted by neuroscientists, where the aim was to map specific brain areas associated with integrating the sight and touch of a looming object. This was in order to enhance scientific understanding of the perceptual and neural mechanisms that are part of multisensory integration. Multisensory integration refers to the study of how information from the different sensory modalities, like sight, sound, touch, smell, self-motion and taste, can be integrated by the nervous system. This is an important part of being human, since a well-formed representation of objects of combining modalities enables people to have meaningful perceptual experiences.
With the new research, subjects were asked to assess the subjective synchrony between a ball that was moving towards them and an air puff delivered to the same side of the face as the ball. The timings were adjusted so that the onset of ball movement and the onset of an air puff were simultaneous (to 100 milliseconds). Each user perceived the two events as one, and perceived the object (ball) passing very near to the face, as if it has ‘touched’ them.
A spin-off from the scientific research is for the device to be developed further for entertainment, like movies, music, games and virtual reality. This will deliver a more immersive multisensory effect near the face and enhance the sense of presence, compared with existing virtual reality technology.
The development of the googles has been detailed in the journal Human Brain Mapping. The associated research paper is “Spatiotemporal integration of looming visual and tactile stimuli near the face.”
