Is it AI synthetic narration or is it human?

So far, not a single professional can tell the difference! These are the surprising results of the launch of Speechki’s book industry quiz about synthetic narration.

Website graphic saying "Quiz: Bet you can't tell a robot from a human!" with the Speechki logo underneath

In November 2021, we participated in the FutureBook Conference. We asked attendees to take our online quiz ‘Real or artificial? You decide!’ Publishers, authors and others close to the publishing industry went to our website (www.speechki.org), listened to 10 pieces of narration, and were asked to distinguish between human narration and Speechki synthetic narration created with artificial intelligence. The results definitely surprised us. Out of 387 respondents, only 8% were able to distinguish between a robot and a human in 10 out of 10 cases!

These results mean a lot to us. If publishing professionals can’t discern when they are listening to synthetic narration, the technology is surely ready for widespread use in audiobooks. Here’s what novelist, poet and spoken word artist Dan Holloway said about Speechki and one of the other companies creating synthetic narration:

“It was a company called Speechki who was giving a talk, and their claim was that their generated voices are so good you can’t tell the difference, and they had a test, and indeed, I think I got 70%. So, yes, it was pretty good, much better. I think we looked at DeepZen a few weeks ago in this column. Their examples of AI narration were really quite clunky and sounded like some of those automatically generated things you get on YouTube, but this was much more sophisticated” — Dan Hollway, “A Report from the FutureBook Conference.”

Speechki wants to make millions more books available in audio format. Human narration will continue to be popular and to grow, but our synthetic narration will allow more publishers to make more books available than ever before. 

We analyzed the results of the first quiz and decided to do some more with it. Four months later, we launched the second version of our quiz with several updates:

  • Men’s and women’s voices
  • Only English
  • Different accents
  • Shorter samples (under 20 seconds)
  • Same texts, different voices

We made this new version of the quiz to check whether our technologies have improved or not. We want you to check your ears and instincts also. Why not give it a try? Go to the link and try to tell a robot from a human! No registration, no contact form, just instant results!

Graphic from the Speechki quiz, showing a YouTube clip (text-only), with two answer buttons below it, saying 'Robot' and 'Human'.

Today we’re happy to announce the preliminary result of the Quiz v2.0. Out of 102 responses, not a single one passed Speechki’s quiz with a perfect score. The average score is only 54% compared to 63% on the previous one. One more interesting stat is that original human voices are detected as synthetic in 57 cases out of 100. 

Of course, this’s the preliminary result, but we can already say that the quality of synthetic narration in Speechki was improved significantly. Now, we’re much closer to the day when real and artificial voices are practically indistinguishable, giving publishers the opportunity to produce more audiobooks and earn more money than ever before. And at the same time, it solves the problem of book accessibility in the audio format, potentially growing the proportion of titles available as audiobooks from the current 5% level to 10%, 20%, 50% or even 99%.

If you want to take the updated quiz yourself, click the link! If you’re a publisher or a self-publishing service, feel free to contact us directly hello@speechki.org to see all the opportunities Speechki gives you, your clients and end-customers.