The new language for robotics is called ‘ComText,’ which is an abbreviation of ‘commands in context’ and it has been developed over several years by MIT’s Computer Science and Artificial Intelligence Laboratory. The language is a step forward in allowing humans to control robots with voice commands, and for the robots to carry out tasks in an adaptive fashion, changing what they do as the environment alters. Taking a simple example, if a robot is told to advance forwards and an object appears in the path of the robot this is not efficient unless the robot can understand that it needs to adapt and alter course.
Taking a second example, if there is a specific tool in a toolbox that is needed and a robot is requested to “pick it up,” this does not work without the robot having a language to engage with. The command to pick up the tool requires the robot to be able to see and identify objects, understand commands, recognize that what the “it” in question actually is; and then to be able to distinguish the tool from other ones of similar shapes and sizes.
To address such complexities, the researchers have developed what they are calling an ‘Alexa-like system’. This computer language allows robots to understand a wide range of commands that require contextual knowledge about different objects and the surrounding environment. The language has been presented in a white paper called “Temporal Grounding Graphs for Language Understanding with Accrued Visual-Linguistic Context.”
According to one of the developers, Dr. Rohan Paul: “”Where humans understand the world as a collection of objects and people and abstract concepts, machines view it as pixels, point-clouds, and 3-D maps generated from sensors.” This means, the researcher explains, there’s a need for a robotic language: “This semantic gap means that, for robots to understand what we want them to do, they need a much richer representation of what we do and say.”
The new language uses episodic memory as the basis, which allows the robot to piece together events and to learn from mistakes. The was language was successfully tested out on a two-armed robot called Baxter. The success rate was 90 percent, for a series of varied tasks. As an example of a business-to-business development, the language has attracted interest and the Toyota Research Institute are planning to adopt it for their car manufacture.
