One of the big reasons that people have pulled away from Siri as a viable option for an assistant, is because “she” hasn’t been able to give really good answers. I remember once, I was trying to find a restaurant, so I asked Siri to give me the names of some restaurants near where I was. Needless to say, she wasn’t able to give me any kind of help in that regard, so I had to turn to Google and look things up the old-fashioned way. Apple, however, has made some pretty big promises recently. Mostly indicating that they were going to improve Siri so that when “she” became available in the HomePod, people would be blown away. But is that the case?
Over the weekend, Loup Ventures shared some results from a battery of tests that it put the HomePod through. The big news? Loup Ventures says that the HomePod was able to understand 99.4% of the commands, but it was only able to answer 52.3% of them. So what does that mean, exactly? Comparatively speaking, Google Home could answer about 81% of the commands, Alexa could answer about 64% and Cortana came in around 57%. Which means, the HomePod is doing worse than Microsoft’s version. Is that even possible? Steve Jobs would be embarrassed.
But I wonder if those numbers for Amazon and Google are even accurate also? To give Apple the benefit of the doubt here, some of the queries that were tested weren’t things that were supported by HomePod. Like navigation, email, and calling. Interestingly enough, when these queries were removed, HomePod was able to answer at a 67% rate. But why can’t Siri answer things related to navigation and email? Sure, maybe calling functionality isn’t quite yet ready to go with the HomePod, but email?
I question Loup Ventures assessment of Google and Amazon in this comparison. I don’t doubt that the Apple HomePod is only providing you with a 53% accuracy rate, but I definitely question where those other two statistics are coming from. To start, Alexa has 15,000 skills. These skills are things like reading recipes, ordering pizza and even calling an Uber. So if the HomePod can’t even read your email for you, why is Alexa ranking so low?
Google certainly has the advantage of their own search expertise, so the percentage for Google makes more sense to me. Some people say that while the Google Home offers more detailed answers than Alexa, but in day-to-day use, it’s not noticeably more capable at performing simple tasks. For example – trying to add an event to your Google calendar is extremely awkward. And while it does have some smart-home integrations and third party “actions”, but nowhere near the number that Alexa does. That said, the Google Home can still understand two commands at once.
All of that said, I am not surprised that the Apple HomePod performs as poorly as it does, because, well, it’s running on Siri. Asking Siri any kind of basic question results in some pretty epic failures. I was under the impression that Siri was getting a big, bad makeover, but clearly, I was mistaken.