According to tests conducted by Carnegie Mellon University researchers, so-termed conversational agents — like the ubiquitous Siri, Alexa and Cortana — are effective at giving users the latest weather or map locations. However, these early forms of ‘artificial intelligence’ are invariably ineffective when asked for atypical information or when follow-up questions are pitched.
These limitations can be overcome, the researchers contend, by adding humans to the loop. Through innovative human/machine hybrid research, the scientists created a new form of conversational agent called Evorus. This is presented by the developers as the first chatbot to use human brainpower to answer a broad range of questions.
This was achieved by simultaneously using humans in the training of the new device’s artificial intelligence. Then, over time as the machine learned appropriate responses, the Evorus device became progressively less dependent on people. This is captured in a white paper titled “Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time.”
To achieve this, Evorus recruits crowd workers on demand via Amazon Mechanical Turk. This is to answer questions from users. Crowd workers are asked to vote on the best answer, and as this is done, Evorus keeps track of questions asked and answered. Through this, and over the course of time, the device starts to suggest answers for subsequent questions.
At the same time, the researchers developed a process by which the artificial intelligence is able to approve a message with less crowd worker participation. According to lead researcher Professor Jeff Bigham: “With Evorus, we’ve hit a sweet spot in the collaboration between the machine and the crowd.”
He adds that, “the hope is that as the system grows, the AI is able to handle an increasing percentage of questions, while the number of crowd workers necessary to respond to “long tail” questions will remain relatively constant.”
The new system remains a work in progress; in the meantime, it is available for download for use by anyone willing to be part of the research effort (see: ‘Talking to the Crowd‘).
