Remember meForgot password?
    Log in with Twitter

article imageAI teaches itself how to understand sentiment

By James Walker     Oct 2, 2017 in Technology
A neural network has taught itself to understand sentiment while completing a mundane text prediction task. The AI had been tasked with predicting the next character in Amazon reviews but ended up analysing sentiment, without any input from its creators.
The neural network was developed at OpenAI, a non-profit AI research firm led by investors including Elon Musk and Peter Thiel. The company documented its discovery in a blog post. The neural network is said to claim "state-of-the-art" sentiment analysis accuracy on the "extensively-studied" Stanford Sentiment Treebank sentiment analysis dataset, even though the AI was only trained to predict the text in reviews.
Sentimental AI
The model was fed data from 82 million Amazon product reviews. It was then tasked with predicting the next character given a chunk of text from a review. The researchers noticed the AI used a relatively small amount of its training data to complete the task.
Digging deeper, they found a "sentiment neuron" had been created that allowed the model to accurately predict the sentiment of the text. The researchers said it was a "surprise" to find the AI had learned how to complete the task by teaching itself sentiment analysis.
Next step in AI development
The discovery could be another step towards the development of unsupervised learning algorithms. These would dramatically accelerate the rate of AI development by removing the need to manually train new models with large datasets. Unsupervised learning could offer increased performance, better predictions and on-the-fly learning of new tasks.
READ NEXT: Samsung becomes latest firm to invest in Canadian AI talent
OpenAI said the "sentiment neuron" isn't exclusive to its neural network. It should show up in other large-scale models that are intended to predict the next step in their input data.
"We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment," said OpenAI. "We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in their inputs."
The research suggests there could already be other neural networks that have learnt sentiment analysis without specific training. The researchers said the discovery of the "sentiment neuron" is a "promising step" towards unsupervised learning but cautioned the circumstances around its creation remain "mysterious."
OpenAI's already conducted some further research into its findings. It has determined that the results aren't so strong for datasets containing long documents or text that diverges from product reviews. There's more work to be done to verify whether the "sentiment neuron" exists in broader sets of data and then ascertain how best to utilise it.
More about Ai, Artificial intelligence, neural networks, machine learning, sentiment analysis
Latest News
Top News