Add to Siri

In iOS 12, users can define their own phrases for launching specific shortcuts via voice. Think of it as the voice equivalent of triggering an action on the Mac using a keyboard shortcut with apps such as Alfred or Keyboard Maestro – only instead of defining a combination of keys that run an action when pressed, you record a sequence of words that, when spoken to Siri, will activate a pre-defined shortcut anywhere in the system.

There are three ways to create a custom phrase for a suggested shortcut. The first method involves visiting the shortcut management screen located in Settings ⇾ Siri & Search. At the top of the screen, you’ll find up to three suggestions for relevant shortcuts; tap ‘All Shortcuts’ and you’ll be taken to a page listing all the shortcuts available for apps currently installed on your device. The Settings app automatically suggests recently used app shortcuts as well as other shortcuts that were previously donated by apps.

Tap All Shortcuts to view all shortcuts donated by apps to iOS 12.

The main list of app shortcuts. Tap the + button to bring up the Siri interface…

…which lets you add a personalized phrase to invoke a shortcut via Siri.

You can view all your previously recorded custom Siri phrases in a separate screen in Settings.

Tapping on the ‘+’ button next to a shortcut brings up a native ‘Add to Siri’ UI that all apps have to use to let you record personalized shortcut phrases. This screen features the name and icon of the app that donated the shortcut alongside a title and description for the shortcut. Moreover, developers can include a phrase suggestion for the selected shortcut to give users an idea for a potential phrase; however, it is always up to the user to decide what they want to record. Developers – even Apple – cannot pre-populate your personalized shortcut list with phrases they’ve chosen; you always have to create a phrase for each shortcut yourself.

Upon tapping the record button, Siri will start listening for a phrase you speak into the microphone and transcribe what it hears. Once it’s recognized the phrase, it’ll stop listening and load a confirmation screen displaying the phrase it heard; at this point, you can either save the personalized shortcut, or tap an ‘Edit’ button to choose between alternative transcriptions. With my Italian accent, for example, a “Show me weight” phrase offered “Show my wait” as an alternative.²²

As an Italian using his devices in English, the accent struggle is real.

Once given a custom phrase, user-configured shortcuts can be viewed in Settings ⇾ Siri & Search ⇾ My Shortcuts. In this list, you can tap on a shortcut to delete it or re-record the phrase if you want to change its voice trigger.

Nice.

The second way to save custom Siri shortcut phrases (as well as to see all shortcuts available for an app) is to visit an app’s configuration screen in the system’s Settings app. In this screen, tap ‘Siri & Search’, then ‘Shortcuts’, and you’ll be presented with a page containing your personalized shortcuts, suggested shortcuts, and shortcuts recently donated by the app. Functionally, adding and managing shortcut phrases works just like in Settings’ main Siri section, but this method has the advantage of coalescing all types of app shortcuts into one screen.

Shortcuts for an app can also be accessed from its native configuration screen in Settings.

Finally, you can save shortcut phrases inside apps. In iOS 12, third-party developers can embed messages and buttons to bring up the Siri UI to record a shortcut phrase for an associated screen or action. While many developers have opted to use plain ‘Add to Siri’ text buttons, I’ve seen apps (such as Mayo’s Daily Dictionary) adopt Apple’s official (and nicer) Siri button (which can be black or white). Developers can also display already-recorded shortcut phrases in their apps’ UIs and allow users to delete or re-record a phrase using the system’s native Siri screen.

Daily Dictionary (left) uses Apple’s official ‘Add to Siri’ button. Other apps I tested opted for custom buttons and a ‘Siri Shortcuts’ section in their settings.

When a shortcut runs in the context of Siri after being triggered by a custom phrase, it behaves similarly to how it’d work in Search or the lock screen, and consistently with regular SiriKit intents. If a shortcut needs to open an app, after hearing a custom phrase the assistant will launch the app into the activity paired with the shortcut. Drafts’ ‘Search draft list’ shortcut, for example, instantly takes you out of Siri and loads the Drafts app in search mode.

However, if a shortcut needs to show you a custom UI or voice response, it’ll run inline within Siri and display its own interface and response without leaving the assistant. The aforementioned Scriptable app can show its custom UI in Siri just like it can when running in Search; if you program a custom voice response in a script, Siri will display the visual shortcut and speak the response too. Similarly, CARROT Weather can provide Siri with a visual forecast plus a (user-customizable) spoken summary; SnipNotes lets Siri read entire notes and uses the visual intent as a tappable button to open an individual note in the app.

Examples of how much apps can customize their intents’ interfaces in Siri. Yes, the one on the right is a video player inside Siri. HomeCam displays a live video feed of a HomeKit camera; the picture of my dogs is presented via JavaScript through Scriptable. Crazy stuff.

There are a few implementation details worth noting about shortcuts that leverage Intents UI extensions. Most notably, custom interfaces can only receive touch events to launch the activity associated with the shortcut in the main app. For instance, you can tap on the shortcut’s snippet for a Drafts note to view the item in the app. However, this also means that visual shortcuts aren’t fully interactive. James Thomson²³ won’t be able to build a calculator shortcut that lets you perform operations with a full-blown calculator UI inside Siri. Content displayed inside visual intents can be animated²⁴, but it’s never interactive.

Developers can include buttons outside the intent’s interface to initiate specific actions or confirm the execution of a shortcut. For example, a food delivery app could embed an ‘Order’ button at the bottom of an Intents UI-based shortcut and perhaps ‘Yes’ and ‘No’ buttons to confirm that the user wants to have food delivered to their house. WaterMinder, a hydration utility, has adopted these confirmation buttons in its shortcuts to log new water entries to iOS’ HealthKit database; after asking users to confirm the shortcut they’ve invoked with a custom phrase, WaterMinder can even update the intent’s UI once the ‘Yes’ button has been tapped and the entry recorded.

WaterMinder’s shortcut includes buttons to confirm the execution of the action, which saves data into the HealthKit database.

Also, a note on syncing custom phrases and apps relying on custom intents. Custom phrases sync between your devices with iCloud, but you have to ensure the same app is installed everywhere for an associated shortcut to work properly on multiple devices. For instance, if you’re using an iPad and invoke a shortcut that was created on iPhone, Siri will give you an error if you don’t have the same app installed. But if you do, the shortcut will launch correctly without having to set it up from scratch.²⁵

As for apps using the new custom intent API: as I mentioned above, they can provide shortcuts with custom responses that are spoken aloud by Siri. This also allows custom responses to be read aloud by Siri on HomePod, Apple Watch, and CarPlay rather than falling back to the iPhone. And as I’ll explain shortly, custom responses are one of the core features behind integration between Siri and the new Shortcuts app.

It’s entirely up to developers to understand the types of shortcuts they want to donate from their apps to iOS 12 and, if necessary, design their Siri intent UIs and responses accordingly. This is not a trivial amount of work, especially for apps that want to go beyond basic launcher shortcuts. But because custom intents and shortcuts are opening up Siri to, effectively, any kind of iOS app regardless of its domain, I’m optimistic that developer adoption is going to be fast-paced and diverse.

The more I think about it, the more I see custom shortcut phrases as the next big step in making Siri a more personal assistant that is unique to each user. As would happen with an actual assistant, shortcut phrases allow users to form their own language over time, creating a personalized set of instructions that only their assistant can interpret and act upon. It’s the equivalent of jargon in a group of friends, but applied to Siri and app actions. The potential accessibility perks are tremendous too: Apple now enables everyone to create custom Siri phrases that can be however long or short they want; this removes the need to find actions nested in apps, multiple levels deep into their navigation.

Apple has been smart in reframing iOS automation around Siri and voice: as I noted above, custom phrases aren’t too dissimilar from keyboard shortcuts. However, spoken phrases are easier to remember – they don’t feel like dark magic to regular users who have never bothered with automation before, and, most of all, they are natively supported across the entire spectrum of Apple products, from iPhones and AirPods to HomePods and Watches.²⁶

I strongly believe that personalized phrases are the first step towards changing the fundamental Siri experience, which is evolving into a personal command log – from one Siri to a million Siris, each uniquely tailored to the user who customized it. Furthermore, custom phrases bring us to the final and most powerful layer of Apple’s automation and Siri strategy in iOS 12: the brand new Shortcuts app.