Smartphone virtual reality headsets such as the Samsung Gear VR and Google Cardboard can offer an immersive experience for substantially less cost than buying a PC-bound Oculus Rift or HTC Vive. The reduced processing power of smartphones limits the detail available in mobile experiences though, preventing control schemes like hand gestures from being implemented.
Israeli gesture control firm eyeSight Mobile Technologies has developed a system that could make this possible in the future. In a video posted to YouTube, the company demonstrated a new technology that uses existing hardware to work out the location of your hand in real-time.
The gesture control is based around your phone’s rear camera, protruding from the back of the virtual reality headset. Using advanced algorithms, eyeSight detects motion in the camera feed and converts it into input signals for virtual reality apps.
The system would let you interact with gestures just by moving your hand up in the air in front of you. The motion would be detected by the camera and interpreted as a gesture. It could make it much simpler to navigate mobile virtual reality experiences, letting you escape the confined menus of current apps.
In its demonstration, eyeSight showed how you could play an augmented reality game using gestures for input. As the game board floats in the air, the player is able to move his finger and “tap” objects to select them and control the game. The phone’s rear camera maps his movements in real-time and is able to detect when he “clicks” something.
eyeSight hasn’t revealed the details of how the technology works. Notably, it remains unclear whether the processing time required to process the camera feed will impact the performance of the app the user is interacting with. There’s only so much power available from a smartphone and as more sensors are added devices can quickly reach their limits.
The technology isn’t ready for public consumption yet. If it does make it onto consumer devices, it should be available on Android and iOS. Because of the simplicity of the solution, it is also headset-agnostic, happy to work with any mobile virtual reality headset currently on the market.
eyeSight appears to have looked at current gesture-tracking systems, typically involving a powerful PC and unique sensors, and then compared it with what’s available on smartphones. The result is the repurposing of existing mobile hardware, usually redundant in virtual reality, into a sensor that is available on every device and operates seamlessly. It could give virtual and augmented reality content a broader appeal.
