Posts tagged with "siri"

Apple’s Machine Learning Journal Posts Paper on How Siri Works on the HomePod in Noisy Environments→

Apple’s online Machine Learning Journal has published a paper on the methodologies the HomePod uses to implement Siri functionality in far-field settings. As Apple’s Audio Software Engineering and Siri Speech Teams explain:

Siri on HomePod is designed to work in challenging usage scenarios such as:

  • During loud music playback
  • When the talker is far away from HomePod
  • When other sound sources in a room, such as a TV or household appliances, are active

Each of those conditions requires a different approach to effectively separate a spoken Siri command from other household sounds and to do so efficiently. The report notes that the HomePod’s speech enhancement system uses less than 15% of one core of a 1.4 GHz A8 processor.

Apple engineers tested their speech enhancement system under a variety of conditions:

We evaluated the performance of the proposed speech processing system on a large speech test set recorded on HomePod in several acoustic conditions:

  • Music and podcast playback at different levels
  • Continuous background noise, including babble and rain noise
  • Directional noises generated by household appliances such as a vacuum cleaner, hairdryer, and microwave
  • Interference from external competing sources of speech

In these recordings, we varied the locations of HomePod and the test subjects to cover different use cases, for example, in living room or kitchen environments where HomePod was placed against the wall or in the middle of the room.

The paper concludes with examples of filtered and unfiltered audio from those HomePod tests. Regardless of whether you’re interested in the details of noise reduction technology, the sample audio clips are worth a listen. It’s impressive to hear barely audible commands emerge from background noises like a dishwasher and music playback.

Permalink

Google Integrates Assistant App with Siri Shortcuts on iOS

Google released an exciting update for its Assistant iOS app today, bringing support for Siri shortcuts and, for the first time, opening lines of communication between the two competing assistants.

Siri and the Google Assistant have historically been unable to work together in any way, but thanks to the opening up of Siri via shortcuts in iOS 12, that changes now. With the latest update, you can set up a shortcut in iOS to immediately, via Siri, trigger any command you’d like to give Google’s Assistant.

Read more

HomeCam 1.5 Adds Shortcuts to View Live HomeKit Camera Feeds in Siri, Search, and the Shortcuts App

I previously covered HomeCam, a HomeKit utility by indie developer Aaron Pearce, as a superior way to watch live video streams from multiple HomeKit cameras. In addition to a clean design and straightforward approach (your cameras are displayed in a grid), what set HomeCam apart was the ability to view information from other HomeKit accessories located in the same room of a camera and control nearby lights without leaving the camera UI. Compared to Apple’s approach to opening cameras in the clunky Home app, HomeCam is a nimble, must-have utility for anyone who owns multiple HomeKit cameras and wants to tune into their video feeds quickly. With the release of iOS 12, HomeCam is gaining one of the most impressive and useful implementations of Siri shortcuts I’ve seen on the platform yet.

Read more

How to Trigger IFTTT Applets with iOS 12’s New Shortcuts App and Siri

Among the actions that didn’t make the transition from Workflow to the new Shortcuts app for iOS 12, built-in support for triggering IFTTT applets (formerly known as “recipes”) is perhaps the most annoying one. With just a few taps, Workflow’s old ‘Trigger IFTTT Applet’ action allowed you to assemble workflows that combined the power of iOS integrations with IFTTT’s hundreds of supported services. The IFTTT action acted as a bridge between Workflow and services that didn’t offer native support for the app, such as Google Sheets, Spotify, and several smart home devices.

Fortunately, there’s still a way to integrate the just-released Shortcuts app with IFTTT. The method I’m going to describe below involves a bit more manual setup because it’s not as nicely integrated with Shortcuts as the old action might have been. In return however, you’ll unlock the ability to enable IFTTT triggers using Siri on your iOS devices, Apple Watch, and HomePod – something that was never possible with Workflow’s original IFTTT support. Let’s take a look.

Read more

Overcast 5: Redesigned Now Playing Screen, Search, Siri Media Shortcuts, and More

Overcast, Marco Arment’s popular podcast client for iPhone and iPad, received a major update today to version 5. While I’ve long praised Apple’s work on their built-in Podcasts app for iOS – particularly since getting three HomePods and leveraging Podcasts’ support for AirPlay 2 – I also recognize the appeal of Overcast’s advanced features and powerful audio effects. Sprinkled throughout Overcast’s release history are design details and enhancements big and small that make it a sophisticated, versatile client for podcast aficionados who don’t want to settle for a stock app. From this standpoint, despite welcome improvements to Podcasts in iOS 12, changes in Overcast 5 make it an even more attractive option that has caused me second-guess my decision to embrace Apple’s native app.

Read more

Tom Gruber, Co-Founder of Siri, Retires from Apple

The Information reports that Tom Gruber, Apple’s head of the Siri Advanced Developments group, has retired to pursue personal interests including photography and ocean conservation. Gruber joined Apple as part of the company’s acquisition of Siri in 2010 along with his co-founders Dag Kittlaus and Adam Cheyer, who previously left Apple in 2011 and 2012 respectively. In addition to Gruber, The Information reports that Vipul Ved Prakash, Apple’s head of search, has left the company. Apple confirmed both departures to The Information.

Siri, which Apple incorporated into iOS in 2011, has been through recent leadership changes as it has fallen behind voice assistants like Amazon’s Alexa and Google Assistant. In 2017, Craig Federighi, Apple’s Senior Vice President of Software Engineering, took over Siri’s oversight from Eddy Cue. Just this past May, Apple hired John Giannandrea, Google’s former Chief of Search and Artificial Intelligence to be Apple’s Chief of Machine Learning and AI Strategy. Last week, Giannandrea showed up on the leadership page on Apple.com, and, according to a TechCrunch story, the Siri team now reports to him.

With all of Siri’s co-founders departed from the company, it will be interesting to see in what direction Giannandrea and the Siri team take Apple’s voice assistant.

Shortcuts: A New Vision for Siri and iOS Automation

In my Future of Workflow article from last year (published soon after the news of Apple’s acquisition), I outlined some of the probable outcomes for the app. The more optimistic one – the “best timeline”, so to speak – envisioned an updated Workflow app as a native iOS automation layer, deeply integrated with the system and its built-in frameworks. After studying Apple’s announcements at WWDC and talking to developers at the conference, and based on other details I’ve been personally hearing about Shortcuts while at WWDC, it appears that the brightest scenario is indeed coming true in a matter of months.

On the surface, Shortcuts the app looks like the full-blown Workflow replacement heavy users of the app have been wishfully imagining for the past year. But there is more going on with Shortcuts than the app alone. Shortcuts the feature, in fact, reveals a fascinating twofold strategy: on one hand, Apple hopes to accelerate third-party Siri integrations by leveraging existing APIs as well as enabling the creation of custom SiriKit Intents; on the other, the company is advancing a new vision of automation through the lens of Siri and proactive assistance from which everyone – not just power users – can reap the benefits.

While it’s still too early to comment on the long-term impact of Shortcuts, I can at least attempt to understand the potential of this new technology. In this article, I’ll try to explain the differences between Siri shortcuts and the Shortcuts app, as well as answering some common questions about how much Shortcuts borrows from the original Workflow app. Let’s dig in.

Read more