In his presentation at the F8 Facebook Developer Conference, Abrash argued that the manner in which optical illusions trick our senses indicate that the human brain is an “inference machine” that struggles constantly to make smart guesses about what is really out there in the real world because of the limited data being made available to it through the physical senses.
In other words, the human brain is challenged constantly to build a complete field of real world experience by literally filling in the blanks in the data set because the full set of information that it needs to interpret its environment is often not available.
Because the brain finds it cannot always rely on the senses to provide a complete set of data, it operates much of the time as an “inference machine” that struggles to overcome the problem of paucity of data by making smart or informed guesses about what is really out there.
As Abrash stated, the brain “reverse engineers” reality “based on the sparse data” it receives.
The observation that the human brain is an “inference machine” that reverse engineers reality from the sparse data supplied by the senses has profound implications for the development of virtual reality technology.
According to Abrash, our increasing knowledge of how the brain constructs experience by filling in the blanks could help virtual reality developers learn how to develop realistic virtual reality systems.
The fact that much of what we call reality is actually the brain filling in the blanks of missing information means that much of what we call physical reality is in fact brain-generated virtual reality.
Thus, according to Abrash, understanding how the brain works could help in the construction of virtual reality experience indistinguishable from what we call “reality.”
We could miss the full implication of Abrash’s comments if we fail to consider them carefully. Those of us with some scientific technical background steeped in the dominant philosophical notions of scientific empiricism have been made to believe that the world is entirely out there and that the brain is merely an instrument for perceiving the given.
Abrash’s thesis implies that this is not altogether true. The world is not entirely out there. Part of the world we perceive is the brain literally patching things up by supplying its own answers to questions about the world that the senses fail to supply.
Thus, to a significant extent, while we are looking forward to a future of virtual reality technology, the human brain is already doing advanced stuff in the field.
[Note: The YouTube video below includes a recording of the livestream of Abrash’s speech at the F8 sumit titled “Why Virtual Reality Will Mattter To You”]
To illustrate, scholars who have studied human color vision point out that despite the richness and variety of our experience of colors, the human eyes have only three color sensors.
As Abrash notes “Our visual data is actually astonishingly sparse and even if we were able to accurately record and process every photon that reaches our eyes, we’d still have too little data to be able to reconstruct the world accurately.”
So how come we live in a world literally awash with variety of colors?
The answer is that our brains are taking the sparse information being delivered by the senses and using it creatively to build the world of rich colors in which we live.
If you are not shocked by this revelation, then you have probably not absorbed the full implication of the fact that a big part of the world we experience is within and not without: We live significantly in a world of virtual reality projection of our own brains.
Abrash expresses the implication without mincing words: “I think it’s fair to say our experience of the world is an illusion. One that evolution has honed to be highly functional in terms of survival and reproduction. VR is about experiencing a virtual world as real [and] experiences are nothing more or less than whatever your mind infers.”
Isn’t it an irony that twenty-first century scientists are being forced to conclusions confirming the philosophical ideas of idealism their twentieth century predecessors, in the era of the ascendance of naive realism, ridiculed as the crass metaphysical superstition of the unlettered masses?
Abrash used the example of the 1999 classic science fiction film The Matrix, to illustrate how virtual reality technology could develop in the future leading to a situation in which technology is able to generate virtual simulations of reality as alternative realities for human experience.
He then proceeded to provide examples of optical illusions to demonstrate his thesis that the brain functions actively as an “inference machine” that generates its own virtual reality to fill in the blanks in the field of empirical sense experience.
Abrash quoted a statement by The Matrix character Morpheus, played by Laurence Fishburne:
“What is real? How do you define real? If you are talking about what you can feel, what you can smell, what you can taste and see, then ‘real’ is simply electrical signals interpreted by your brain.”
With reference to the red and blue pills Morpheus offered to The Matrix protagonist Neo as options to either exit or remain in The Matrix, Abrash presented several optical illusions to demonstrate his thesis that much of what we observe as physical reality is virtual reality generation of the brain.
In the image above, we perceive the first pill as blue and the second as red although both pills are grey because the brain uses background information to guess the color of the pills. And even when the viewer knows that the pills are actually grey his brain continues to deceive him, showing the pills as blue and red respectively.
He demonstrated another optical illusion using a Rubik’s cube in which tiles of the same drab grey color appeared as blue against a yellow background and yellow against a blue background.
Abrash’s conclusion is astounding if the reader can pause to absorb its implications fully:
“Your visual system isn’t interested in whether the photons coming from a tile on a random image are red or blue or grey. Knowing that didn’t keep anyone from being eaten by lions on the Savannah. What it is interested in is identifying potentially relevant features, in the real world.”
Making a reference to the recent blue-black or white-gold dress optical illusion that went viral online, Abrash said, “Our visual system takes its best guess and sends that to the conscious mind. The way that the brain compensates for the limited data it receives is by maintaining a model of the real world that it constantly updates as new data comes in.
“And it is that model, not the real world, that you experience and trust implicitly. We are inference machines, not objective observers.”
The implication of the insight that Abrash offers is that contrary to what we have believed there are no hard and fast dividing lines between virtual and physical reality, at least as far as phenomenal consciousness experience is concerned. Even while we plan for a future of virtual reality experience, recent research into optical illusions shows that the part of what we call reality is brain-generated virtual reality.
Abrasch continued: “Morpheus made two critical points with his sentence: our conscious minds never actually interact with the real world and that we interact with sensors on our eyes, ears and tongue, and throughout our body. This is just a very small subset of the real world.”
Despite Abrash’s continued reference to the idea of a “real world,” many thinkers, such as the quantum physicist David Bohm, in his book Wholeness and The Implicate Order, have questioned the persisting assumption that there really is a distinct “real world” out there that the brain attempts to replicate constructively using data passed to it by the senses.
Bohm notes that developments in physics, beginning with Einstein’s relatively theory, first proposed in the early twentieth century, indicate that contrary to the assumptions underlying the Newtonian paradigm that dominated 19th century thought, which conceives of rigidly delineating borders between the observer-consciousness (subject) and the physical environment (object), there is no absolute physical space-time system out there which the mind-brain system struggles to orient itself with. What exists is a fluid continuum in which the observer-consciousness (subject) and the physical universe (object) interact as aspects of a unified whole.