Intelligence
For many of us, iOS devices are the most important computers in our lives. With iOS 9, Apple is rolling out a series of features for proactive recommendations and intelligent suggestions aimed at making the devices we use every day smarter, more contextual, and personal. The results are mixed, but the beginning of something new is afoot.
The source of all of iOS 9’s proactive and intelligent features is our data and daily routine. By using information we store in system apps such as Mail and Calendar and by observing our habits, iOS 9 can discover patterns in the apps we use, when and where we tend to use them, and it can offer shortcuts to show us what we’re most likely going to need next. In broad strokes, this is at the core of Apple’s proactive initiative: by learning from our habits and data, iOS can be a more helpful assistant in everyday life. Unlike similar efforts by other companies (namely Google), Apple has prioritized user privacy when building these functionalities into iOS 9, a design choice with a deep impact on what the OS is capable of suggesting.
Intelligent and proactive recommendations are scattered throughout iOS 9 with shortcuts in various places. Some of them are labeled as Siri features, while others are new options in existing apps that use content from other apps to save time with suggestions.
As a starting point, the new Search page features Siri suggestions for contacts and apps. Displayed at the top of the page and replacing the old contact shortcuts of the iOS 8 app switcher, these shortcuts aren’t indicative of contacts explicitly marked as favorites or apps you’ve recently used. Rather, these suggestions are based on what iOS thinks you’re going to need.
Recommendations are informed by different variables, such as frequency of use, time of the day, day of the week, current location, and other patterns the OS spots and that are used to build up suggestions for apps and people. There’s a chance you’ll see an app you only use on Thursdays and friends you only contact during the weekend. One time, I got a suggestion for my Shopping list in Reminders, and I later realized it was because I was at the grocery store and iOS had memorized the list I frequently used when shopping there.
I also saw a recommendation for this review’s EPUB in iBooks (with a read completion status) when I was assembling the eBook and constantly checking it out in the app.
When working as advertised, Siri suggestions are handy because they bring serendipitous discovery to the Search page. And because they’re not a static set of favorites but a dynamic, continuously updating list of shortcuts that learn from you, they adjust alongside your routine and what you’re likely to do next.
For example, I use the Do Button app every night before sleep to tell my girlfriend the exact time I went to bed with an email. Now, iOS 9’s Search page shows me a shortcut to that app every night between 3 and 5 AM, when I typically go to bed. I’ve seen suggestions for Google Maps at specific places where iOS knew I was going to start navigation in the app, and a couple of friends pop up on Saturdays because I tend to text them and ask them to go out for dinner (when tapping a contact, iOS displays phone, message, and FaceTime actions for it). iOS 9 has picked up some of my habits, and when I come across a suggestion that is accurate, I’m glad iOS is helping me save time.
That’s not always the case, though. The patterns that I described above are fairly easy to spot as they’re repeatable, discrete routines that stand out from everything else. But I use my iOS devices all day every day, and I switch between apps and conversations a lot. The average result is that, for me, Siri suggestions in the Search page mostly are a random selection of shortcuts with the occasional gem that appears at regular intervals when needed. That’s the problem with a general purpose suggestion feature based on “patterns”: when you use your iOS device too much (as I do), app and contact suggestions tend to feel like a lottery. They can’t spot too many distinguishable patterns just by looking at which apps I launch.
Apps like Slack, Twitterrific, and Twitter are always listed in my Search page, but that’s because I always use them. When I don’t see Twitter and Slack, I see a repeat of apps I’ve recently used – the same entries from the app switcher. Do these make sense as persistent suggestions intermixed with shortcuts based on specific times of the day and locations? Wouldn’t it be better to identify the apps I’m constantly using and display them in some kind of separate Top Hits view? Of course I know I’m going to be reading Twitter and Slack. There’s no point in iOS telling me to do so.
Another issue, I believe, is that Siri suggestions in this screen don’t have any explanation attached to them. iOS 9 doesn’t say “Good morning, here’s what happened in Slack last night and I think you’re going to need CityMapper next because you take the subway on Tuesdays”; it just brings up a bunch of shortcuts, leaving it up to you to figure out if they’re useful or not. Sometimes, they are, and it feels nice. Most of the time, they are too generic to warrant a top spot in the Search page, and I think of turning them off entirely. I haven’t yet because I’ve seen how they can be useful at times, but I’d like them to be more than recently used apps.
Along the same lines, the Search page offers shortcuts for Nearby businesses in Maps. Besides the fact that, for my area in Rome, Apple’s business database continues to be outdated and lackluster, I don’t understand the kind of Maps suggestions iOS gives me.
I would expect these recommendations to account for my habits (as tracked by iOS’ Frequent Locations feature) and likelihood of needs to show me businesses relevant to my routine and time of the day. Instead, iOS’ Nearby section has shown me all sorts of business suggestions at the most disparate times: convenience stores at 9 PM alongside “Nightlife” POIs37; coffee shops and restaurants at 3 AM38; “Fun” and “Transport” suggestions in the afternoon, which usually don’t go well together.
In three months, I have never found the Nearby suggestions in the Search page to be useful. The categories are too broad for me to understand at a glance whether the place I’m looking for is “Fun” or “Nightlife”, and the time of the day when they are displayed is usually not in line with what I do on a daily basis. I would prefer iOS to provide me with practical advice for individual places I frequently visit, such as the current traffic to get to my neighborhood supermarket or weather conditions at my favorite beach. Alas, this kind of detail and personalization isn’t available for Nearby suggestions, and that’s disappointing.
The other system-wide proactive mechanism of iOS 9 is standalone app recommendations. In this case, iOS will suggest an app to launch in the same area where Handoff for apps is displayed in the Lock screen (bottom left) and app switcher (at the bottom in iOS 9).
These shortcuts are, like the Search page, accounting for different variables to recommend an app you’re likely going to need. Because of their placement in the UI, they can be more easily noticed when your device enters a scenario that iOS identifies as a pattern.
In addition to time of the day and location, iOS 9 can monitor Bluetooth and audio connections and guess which app you may need when that happens. If you tend to open the Music app after plugging in your EarPods, iOS will bring up the Music icon and media controls in the Lock screen, or it’ll display a shortcut in the app switcher telling you that you can open Music because an audio connection has been detected (I like how these suggestions come with an explanation). Or, if you like to watch Netflix with your Beats Wireless on, iOS will also spot that pattern and recommend Netflix as soon as a Bluetooth connection is established.
The same variable can lead to different recommendations in different times of the day. Listen to audiobooks on your way to work in the morning but to podcasts when going back home? iOS 9 will show different apps for those two scenarios, learning and adjusting over time.
Apple also added contextual awareness support for getting in and out of the car in iOS 9, combining that with proactive suggestions. This isn’t well documented by Apple, but as seen with Reminders, iOS 9 has the ability to recognize user presence in a car by looking at connections to generic car Bluetooth devices as well as CarPlay. The car becomes another dimension for smart suggestions in iOS 9, which can give you app shortcuts based on what you do – such as listening to podcasts or music when driving – but also traffic notifications for where you’re most likely going. The idea of using the car as another layer of user patterns is an intriguing one, and while I couldn’t test this because I don’t have Bluetooth in my car, impressions from hundreds of users I polled on Twitter were positive.
I find individual app suggestions to be nice, and generally more timely and relevant than what I see on the Search page. While not revolutionary, it’s nice to be able to quickly open Music or Overcast when my Beats are connected via Bluetooth, and I’ve been surprised by how iOS picked up that I was going to need Google Maps or Nuzzel at specific times of the day, putting them on the Lock screen. As more and more sensors fill our homes and clothes going forward, I fully expect iOS to gain support for deeper context recognition – imagine suggestions powered by HomeKit devices, proximity to an Apple TV, or beacons.
That’s not to say that app suggestions in the Lock screen and app switcher are perfect: I’d still like to see more targeted suggestions for Music, as iOS hasn’t figured out that I like to listen to Death Cab for Cutie every night before bed. The granularity and timeliness granted by audio and Bluetooth connections have led to more useful app suggestions in my experience, but they can improve.
Next up is Mail, which is used in iOS 9 as an information database in three ways. In Contacts, you can search for people found in Mail and add them as new contacts with some fields already filled-in, or you can add new email addresses to an existing contact as iOS will match the same person between Contacts and Mail. This has been useful to update old contacts with new email addresses from my own correspondence.
Secondly, when receiving a phone call from a number that’s not in your contacts (we all dread those phone calls), iOS 9 tries to discover who it is by looking into Mail messages. I only had that happen once, but it worked as expected.
Last, iOS is more proactive in offering to create new calendar events or contacts from messages that contain such information. A new banner displayed at the top of a message provides a shortcut to create new events and address book entries with one tap, which is a more visible option than iOS’ existing support for smart data detectors in message bodies. iOS 9 can also detect events from messages that contain flight details or restaurant reservations and put them in Calendar for your consideration; you can choose to turn off event suggestions based on Mail in Settings.
Using Mail as a repository of information for other apps is an interesting idea: despite its somewhat archaic nature, a lot of our communications and notifications still come through email, and scanning Mail to bring up shortcuts and suggestions seems like a good idea to me (and I would like to see more of it). If anything, these features highlight the benefit of using Apple’s native Mail and Calendar apps over third-party clients, which will get none of these integrations. I wouldn’t be surprised to see users keeping Mail fetching messages in the background without using it just to take advantage of suggestions.
iOS 9’s proactive suggestions for Contacts and Calendars don’t stop at Mail. When I was on vacation with my girlfriend, we used Apple Maps in Positano to browse restaurants nearby and call them to make a reservation. I didn’t have their phone numbers in my address book, but I noticed that iOS 9 used the restaurant’s name (taken from Yelp, I’d guess) in the Recent Calls screen, which was a nice touch.
In Calendar, events that contain an address now offer the option to use a Time to Leave notification that will send an alert when it’s time to leave for an event depending on your current location and traffic. When receiving the notification, you can snooze it for later or tap it to see directions and get going.
This is another interesting idea, but it hasn’t worked well for me in practice. Events that I knew would take me 10 minutes to get to a location consistently sent Time to Leave notifications 25-40 minutes ahead of time. Not even after “teaching” the system that my driving style and traffic weren’t as imagined by its intelligence did iOS learn that there was no need to send a notification 40 minutes early. I’ve wondered if the wiggle room between the notification and event time could be cultural: perhaps Americans like to arrive early at their events and don’t mind waiting 20 minutes while sipping on their ventis. But in Rome, there’s no such thing as arriving early or spending 20 precious minutes to wait for someone else. Time to Leave is a cool idea, not suited for my habits and local traffic.
There are more intelligent and proactive features throughout the Search page and the OS, but I’ve found their realization to either be dull or their impact to be minimal. You can ask Siri to bring up photos from specific time period and albums. The Search page has a News section at the bottom, which, as a European using an iOS device with a US region format, I found to be an unappealing mix of news about presidential elections, football, and TV spoilers; in theory, iOS should be able to display news relevant to my location, but given that they’ve always been American-heavy topics, I imagine iOS is basing its news collection skills on the user’s region format. You can also ask for weather, calculations, and sports results in the Search page, but only calculations and unit conversions worked for me (I’d call them an expected utility more than an intelligent feature).
Our devices are becoming smarter and more context-aware every year, but Apple’s foray into intelligence and proactive suggestions doesn’t substantially alter the user experience of an iOS device. Instead, what Apple has put together is a mix of sometimes-working, nice-to-have additions that feel unfinished or that are poorly realized. Suggestions in the Search page leave much to be desired, with useful patterns that are obfuscated by generic and unmotivated shortcuts. Standalone app recommendations based on location and audio connections and Mail’s intelligent scanning is where Apple’s vision feels clear and coherent, with delightful discoveries that can save some time every day.
If you’re accustomed to the level of automated intelligence in Google services such as Google Now and Inbox, iOS 9 won’t offer that kind of experience. The Search page is far from the uncanny precision of Google Now when plugged into all of your Google data, and Mail’s new shortcuts pale in comparison to the automated processing and organization tools found in Inbox.
This is by design: while Google can pull it off thanks to their expertise and investment in looking at patterns across all of your data (which happens to be their business model), Apple has decided to prioritize user privacy as much as possible. That’s why proactive suggestions are (mostly) processed directly on-device, with Apple never syncing any of your usage patterns between devices or matching data from one Apple service to the other to build a more complete profile of you.
It comes down to personal preference and the level of potential creepiness you allow in your computing life. Google tends to deliver impressive intelligent features and shortcuts, at the expense of a wealth of data given in return. Apple’s efforts in iOS 9 are more modest in scale, but deeply integrated with the OS and built with privacy in mind.
It’s not about arguing who’s better; it’s about choosing what works better for you. Are you comfortable with Google’s impressive intelligence and suggestion tools knowing that they need as much data about you to power them, or do you prefer Apple’s more private but also less effective approach?
My stance on these issues has changed a lot since two years ago, especially after trying Inbox, Google Now, and having to go back to Google Apps’ Gmail due to slow IMAP sync and search. While I conceptually don’t like the fact that my data is being used by an army of algorithms, the service I get in return is useful and it lets me work faster every day. When it comes to faster work, measurable efficiency trumps ideological stances. As an Italian movie once said, you can’t buy groceries with ideals.
In Google’s Inbox, search is crazy fast, the app detects sentences that look like reminders, and it categorizes emails for me. When I was in San Francisco earlier this year, the Google app automatically pulled in my flight and hotel information from Gmail, displayed it with handy cards in the main view, and it figured out when I arrived at SFO and showed me weather reports and currency exchange rates.
That was pretty amazing and useful, and it’s not something that iOS 9’s intelligence is able to provide just yet. And I have to wonder if it ever will, given that what Google does – the depth of its user tracking and cross-service integration – could only be possible with constant, cloud-based data collection that doesn’t fit with today’s Apple.
Imagine, though, if Apple was willing to look for patterns inside apps, understanding what we write in private communications and what we search for to spot more useful patterns than app launches and EarPods connections. Would they have the skills required to build such intelligence? Would they want to?
Ultimately, it’s not fair to compare iOS 9’s intelligence to Google services: Google will never have this kind of access to device hardware and daily user patterns. iOS 9 delivers on small, periodic proactive enhancements that are meant to save time and surprise users. Their impact is not dramatic: some of them are nice shortcuts, but by not deeply aggregating user data from multiple sources, most of them are generic and stale.
Caught between the tension of respecting user privacy and deepening data collection for proactive features, will Apple be able to ship more useful suggestions in the future? And how will they build it all?
There’s a lot of work to do. It’s up to Apple to figure out what their ideals can allow.
