Images via Pixabay/Victor Tangermann
Tech Talk

Google Says Its Chatbot Is Capable of Near-Human Conversation

byKristin Houser
1. 29. 20
Images via Pixabay/Victor Tangermann

Meena's responses are both logical and specific.

Better Tay

If you’ve ever tried to have a conversation with a chatbot, you know that even today’s state-of-the-art systems aren’t exactly eloquent, regularly doling out nonsensical or painfully generic responses.

Now, though, Google has created Meena, a chatbot it says is better than any other it’s tested — a claim the company supports using a new metric it developed specifically to measure an AI’s conversational abilities.

Making Conversation

After creating Meena, a process detailed in a paper published on the preprint server arXiv, Google needed a way to evaluate the chatbot. To that end, it developed something it calls the Sensibleness and Specificity Average (SSA).

To compute this metric, Google asked human workers to conduct about 100 free-form conversations with Meena and several other open-domain chatbots. Each time the chatbot responded, the worker had to answer two questions about the response.

Advertisement

First, did it make logical and contextual sense within the conversation? If yes, they then had to answer the question, “Was it specific to the conversation?” This was to weed out any generic responses — for example, if the human wrote that they liked tennis, and the chatbot responded, “That’s nice,” the response would be tagged as “not specific.”

Almost Human

Google determined that an average human would achieve an SSA score of 86 percent.

The other chatbots in the team’s study scored between 31 percent and 56 percent. Meena, however, scored a 79 percent — putting the AI closer to the level of conversation expected from a human than another chatbot.

READ MORE: Meena is Google’s attempt at making true conversational AI [VentureBeat]

Advertisement

More on chatbots: Taylor Swift Reportedly Threatened Microsoft Over Racist Chatbot


Futurism Readers: Find out how much you could save by switching to solar power at UnderstandSolar.com. By signing up through this link, Futurism.com may receive a small commission.

Share This Article

Copyright ©, Camden Media Inc All Rights Reserved. See our User Agreement, Privacy Policy and Data Use Policy. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with prior written permission of Futurism. Fonts by Typekit and Monotype.