Loup Ventures, a US-based venture capital firm, ran a series of Siri tests on the HomePod to evaluate the assistant’s capabilities on Apple’s new speaker. After 782 queries, Siri understood 99% of questions but only answered 52% of them correctly – meaning, Siri on the HomePod failed to answer one out of two questions. I’d love to see a full data set of the questions asked by Loup Ventures, but, overall, it doesn’t surprise me that the Google Assistant running on the Google Home speaker was the most accurate in every category.

While Apple has clearly a lot of work ahead for Siri on the HomePod (this was the consensus of all the reviews, too), it also appears that Siri simply performs worse than other assistants because it doesn’t support certain domains. Here’s Gene Munster (whom you may remember for his Apple TV set predictions), writing on the Loup Ventures blog:

Adding domains will quickly improve Siri’s score. Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct. This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%). We know Siri has the ability to correctly answer a whole range of queries that HomePod cannot, evidenced by our note here. Apple’s limiting of HomePod’s domains should change over time, at which point we expect the speaker to be vastly more useful and integrated with your other Apple devices.

Adding new supported domains would make Siri’s intelligence comparable to Alexa (at least according to these tests), but Apple shouldn’t strive for a honorable second place. Siri should be just as intelligent (if not more) than the Google Assistant on every platform. I wonder, though, if this can be achieved in the short term given Siri’s fragmentation problems and limited third-party integrations.