Drew Gottlieb detailed his proof of concept system in a blog post this week. The software developer had the idea after watching a friend use Tilt Brush, Google’s virtual reality painting app for the HTC Vive. His friend asked him to look at what she was making, requiring her to focus more closely on the object and him to squint into the monitor.
In Gottlieb’s words, “we can do better than this.” He set out to create a new system that would let him see his friend’s artwork floating in the room around him, like in augmented reality. This isn’t possible with just the Vive – it’s a closed virtual reality headset that leaves you unable to see your surroundings.
To create the concept, Gottlieb added Microsoft’s HoloLens augmented reality headset to the Vive setup. HoloLens overlays its content on the physical world. Now the objective was feeding data from the Vive to the HoloLens, enabling content in the virtual environment to be visualised in the real world.
The HoloLens has to be calibrated so it can align itself with the Vive’s motion-tracking control. The HoloLens speaks to the user and guides them into intersecting the controller with a holographic counterpart. According to Gottlieb, the solution works better than expected, covering three degrees of position and one degree of rotation.
In a series of videos, Gottlieb demonstrated the prototype system in action. It removes the limits of the virtual reality headset, letting you draw 3D art in the middle of your living room rather than an empty black space. It also opens the door to collaborative experiences. Anyone else with a HoloLens can see the art and participate in it, without squinting at low-resolution renderings on computer monitors.
Gottlieb has open-sourced the code that made the project possible, enabling other developers to develop similar systems. While it’s far from being a robust solution, it works remarkably well and could hint at the future of mixed reality environments.
“I have no doubt this kind of mixed space will be a big part of the future, especially for creative industries,” said Gottlieb. “As virtual and mixed reality become stronger platforms for content creation, it’s only inevitable that they’ll be able to interact on a whim.”
Gottlieb now works on the team behind Tilt Brush but began creating his project before becoming a Googler. He hasn’t said if he’ll continue working on the system going forward. The current prototype was built in just four days, involving the creation of the pairing assistant, Vive to HoloLens connection and a rudimentary 3D painting app. The complete code can be found on GitHub.
For more videos of the system in action, see Drew Gottlieb’s blog.
