Virtual reality painting simulations aren’t new. Google Tilt Brush has been held up as an example of virtual reality at its best, allowing you to draw whatever you like in a vast open space. Project Wetbrush is something different though. It wants to be a painting experience comparable to using real materials, providing feedback sufficient for drawing on 2D surfaces with virtual 3D paint.
Painting with Wetbrush is done with a stylus, just like on current graphics software. When you touch the screen, the software does more than add a dot of colour to the canvas though. Instead, it accurately simulates the characteristics of real oil paint, calculating the thickness, flow and viscosity of each brush stroke to create art that looks like it was created in real life.
Wetbrush makes it possible to show texture in the oil paint, the marks that define real artwork and add character to individual scenes. The texture added is calculated based on the brush speed and angle, like with real materials, and is affected by attributes such as paint drying time.
The software powering the project was developed by Adobe Research. Accurately simulating oil paint is no easy task for a computer so Adobe partnered with NVIDIA to gain access to some of its most powerful graphics cards. These are well suited to the demands of real-time simulations.
NVIDIA describes Wetbrush as a full 3D painting simulation. Current painting and drawing tools are purely 2D, creating dots on a screen. Wetbrush is “completely different,” allowing for multiple levels of thickness, depth and texture that combine to create an experience that could be real.
“Oil painting on an actual canvas is full of complex interactions within the paint, between the brush and the paint, and among the bristles themselves. Project Wetbrush simulates all this in real-time, including the complexity of maintaining paint viscosity, variable brush speeds, color mixing and even the drying of paint,” said NVIDIA. “The bottom line is that it’s not easy to build a digital oil painting tool that lets artists paint so fluidly and naturally that they can ignore the technology and simply immerse themselves in their art.”
NVIDIA demonstrated Wetbrush at the SIGGRAPH show in California this week. It has also prepared a detailed technical paper with Adobe that goes in-depth on how Wetbrush creates the most immersive digital painting experience available.
Adobe and NVIDIA think they could get much more from the project in the future. Wetbrush is already the most advanced system of its kind but the partners see the potential to make more optimisations that could further improve the experience. By using GPU-accelerated deep learning, brush dynamics could be made even more responsive and realistic.
Later versions of Wetbrush could use machine learning capabilities to learn from themselves. NVIDIA proposed a database of realistic high-quality painting and brush strokes could be created. This resource could then be used as training material for future oil paint simulators.
Wetbrush could be used to introduce people to art without having to buy and waste physical resources. The hardware required to power it is prohibitively expensive today but in the future it could be used as an introductory and teaching tool, in the same way pilots begin their training in flight simulators. Advances in machine learning and virtual reality allow for new experiences we’ve never encountered before, with Wetbrush being just one possibility.