Photos
The Photos app in macOS Sierra has received a nice new set of features. Most of these are making use of what Apple is calling “computer vision technology” to run powerful object and scene recognition on your photo library. The results are impressive, and are being used in a few of the new features that have been added to Photos.
Memories
The biggest new addition, based on Apple’s announcement of the update at WWDC, is Memories. Memories is an attempt to intelligently gather together groups of related photos and videos into a “memory”. The goal is to allow you to relive certain events or periods in your life through the photos and videos you took of them.
Each “memory” will have a label such as ‘Best of Last Month’, and consists of a slideshow and a set of photos and videos. Double-clicking a memory will open it, showing the slideshow at the top and a summary of the photos making up the memory below. Memories will try to pick the “best” of all the photos that are drawn into the period. A Show All button will allow you to expand the summary of the memory to show all of the photos and videos included in it.
Underneath the photos and videos of the memory, you’ll find a ‘People’ section, which includes any people that Memories has determined to be in the photos above. The people displayed come from the new People feature in the app, which I’ll discuss below. If Photos has not identified anyone, this section won’t be present.
Below the People section you’ll find a map showing all the places where photos and videos from the memory were taken. Below that is a “related” section, which will show even more memories that you can look at.
At the very bottom of the page you’ll find options to delete a memory completely or add it to your “favorite memories”. The latter will create (or add to if you’ve already created it) a new “Favorite Memories” album, where you can see all the memories you’ve marked as favorites.
While Memories isn’t a feature that I use very often, I’ve been impressed by what it has surfaced over the last few months. Memories seems to generate new content sporadically, so if you check in every couple of weeks you will likely see new entries popping up (depending on how often you take pictures). I haven’t found the slideshows particularly compelling, but the groupings of photos and videos have been a fun way to look back on old times when I’m in the mood for that kind of thing.
People
The next “new” feature of the Photos app is the People section. While Apple is billing this as new, it’s really just an updated version of the Faces feature which has been in Photos since its beginning. If you have ever done any work to categorize pictures in Faces in the past, those have all been moved to here.
At the top of the People interface you’ll find a grid of people you’ve marked as “favorites”, and below those is the grid of the rest of the people that you’ve defined. When you double-click a person’s entry, you’ll open up an interface similar to Memories, showing all the photos of the person at the top. Below these will be a section showing other People present in the photos, below that a Places section showing where the photos were taken, and below that a Related section showing more Memories.
At the very bottom of the page are options to confirm additional photos of this person, favorite or unfavorite them, or add the entire group of photos of them as a memory. That last option will create a new entry in the Memories section of the app which is titled by the name you gave the person in the People section, and opens to the People interface.
If you select the ‘Confirm Additional Photos’ option, Photos will initiate a search of all of your photos looking for the person in question. This shouldn’t take more than a minute or two, but I’ve seen it get stuck a few times. If it takes longer than that to display the first photo, try hitting ‘Done’ in the top right and then hitting the button again.
Once it works, you’ll see photos show up one at a time, with a highlight circle around one of the faces in the photo. You can confirm whether the face has been correctly recognized by choosing Yes or No at the top. Keep choosing until you’ve gone through all the photos the app has recognized, or hit Done whenever you want. Also of note is that if the app gets a face wrong but you click Yes or hit return before you realize it, you can just hit Undo to go back to the previous photo and fix the mistake.
Adding new faces to the People interface is done at the bottom of the main interface, underneath the grid of current People. Clicking the plus button there will display a grid of recognized, but not yet categorized faces. You can then select one face and choose ‘Add’ to create a new Person based on that face, or select multiple faces and choose ‘Merge & Add’ to combine all of them into one new person. Once you’ve added a new person you’ll be back on the main People interface, and can then click underneath the new addition to add a name. If you already have another entry for the person you’ve added, just give them the same name (it should appear in a drop down as you type) and Photos will ask you if you’d like to merge the new photos into the existing Person entry.
Photos will automatically select a group of photos that it thinks is of the person in question when you create an entry for them. The more Photos you manually select when you add the original entry, the more photos the app will connect to that Person. Similarly, when you confirm a batch of photos of someone within the Confirm Additional Photos interface, the app will connect more photos than just the ones you confirmed. Every photo added increases Photos’ knowledge and ability to accurately add even more.
In playing with the interface, I’ve found that Photos seems to be fairly conservative with its guesses for confirming additional photos of people. It rarely proposes a photo that is not of the person in question, but because of this it also seems to miss a lot of photos of that person that it isn’t sure enough about. The best way I’ve found to get through this block is by adding another batch of photos that you select manually of the person, then merging those into the existing entry. This gives Photos more data to work with, and in turn it will start proposing more photos for you to confirm.
Overall, I’m not sure whether the facial recognition algorithm behind the People feature is any better or worse than the one behind the Faces feature last year (I’m pretty sure it’s at least somewhat different due to Apple’s computer vision technology being a brand new thing, but it’s hard to know for sure without someone in Apple telling us). At the very least it is comparable. Still, the interface for the People feature is a significant improvement over the clunky Faces interface from last year, and makes adding and merging new batches of photos for the same person far easier. Since compiling as many faces as possible for each person is pretty much the whole point of the interface, I give this section of Photos a big thumbs up compared to its old implementation.
Places
The new Places feature has been mentioned several times above, but basically consists of a new map interface that can now be found all throughout Photos. The Places map exists within the Memories and People interfaces, and will display the locations on a map of every photo in the selected memory or of the selected person, respectively.
If you open up any particular photo from anywhere in the app, you’ll see a Details button in the top right corner, and clicking on this button will show all the data about the photo. This includes which People the photo has been found to contain, as well as the Places map that will show where it was taken. In the top right above the map will also be a Show Nearby Photos button, which will inflate the map interface to fill the Photos screen and display all the pictures taken nearby to the photo that was selected.
Places is not really a new feature for Photos, as previous versions did separate groups of pictures by location as you scrolled down through your All Photos view. The utility of Places, however, is in exposing that location data in a far more prominent way than it was before. Now you can much more easily see a great visual representation of where your photos were taken, and it is more or less ubiquitous throughout the app.
Search
The final feature of Photos is the one I find to be the most useful. The search functionality of the app has been greatly improved in quality as well as in scope. This is where that fancy “computer vision technology” is truly coming into play.
The new search in Photos for macOS Sierra allows you to search for the content of your photos rather than just the metadata about them. For instance, if you want to see all of your photos that include trees, just search for ‘Trees’ and you’ll get them. This kind of computer vision is extremely difficult to pull off, but Apple has done a fairly decent job of it here.
When you first install macOS Sierra and open up the Photos app, it will scan your entire Photos library and start running algorithms and calculations to parse the objects within all of your photos. You should note that this process can take quite a while (hours or even days, depending on the size of your library), and these features will not be functional until it has been completed.
Once the process finishes and your pictures are ready to search, it’s worth noting that you can’t search completely freely. Rather, your searches are limited in scope to the categories that Apple’s scanning has identified within your photos. By extension, any categories that Apple does not scan for simply will not exist.
When you type in a search term, Photos will start showing suggestions for what you might be typing. If you type something that doesn’t exist as a category, you’ll get “no results” and be unable to run the search at all. When you do search for something that Apple has scanned for and found, you’ll see all of the photos that match the term shown in the window below. Unfortunately, there’s no way to see a list of all of the available search terms for your photos. As such, you just have to guess and check and see if the term is there.
In real world usage, search is at its most useful when there’s a specific photo that you’re trying to find. The hard part now is training yourself to remember that the feature is available. I’ve had multiple times over the last few months where I want to find an old photo and I spend forever looking for it manually, only to remember at some point after I’ve found it that I could probably have just run a search and surfaced it in seconds. If you think of search first and the photo isn’t categorized by anything you search for, you’ve only really lost a few seconds. If the photo does show up from the search, you may have just saved yourself several minutes of time for that single task. It’s a very powerful tool when it works right. Based on my own personal library, while it isn’t 100% accurate — or really even close to that — it’s accurate enough as to be a very useful tool.
Overall, Photos for macOS Sierra packs some interesting new features which push the application forward. This year’s update to the app focuses strongly on surfacing new data from our photos that was previously much more difficult to find. We can discover old memories, more easily compile photos of particular friends and family, and see exactly where we’ve taken our photos. We can even search for objects within them. As far as year-over-year updates go, this one seems like about as much as we could ask for.
