Drag and Drop

If we had to distill iOS 11 down to a single, all-encompassing enhancement, it would be drag and drop. This feature is the embodiment of several ideas with far-reaching ramifications throughout the whole system.

As a way to copy data between apps, drag and drop is, at a basic level, a classic Mac feature reimagined for multitouch. For decades, Mac users have been able to move content from one onscreen location to another using an abstraction of their hand – the cursor – to drag items and drop them somewhere else. Drag and drop was a perfect match for the Mac’s graphical interpretation of a desktop: if you’re looking at virtual objects laid out on a desk or filed in folders, it’s only natural to assume you can manipulate them directly. The modern GUI was founded on this principle: that users can interact with the computer like they’d interact with a workspace in the physical world.

That’s why the lack of a system-wide drag and drop framework on iOS always felt like a strange omission: as a successor to OS X built for direct content manipulation, iOS was an even better candidate for drag and drop than the Mac. On iOS, you don’t control an accessory that communicates with the interface: your finger is the ultimate accessory. And we have ten of them. Wouldn’t multitouch be the definitive expression of drag and drop?

Apple’s answer in iOS 11 is a resounding “yes”, but, predictably, bringing drag and drop to iOS never was a matter of flipping a switch to magically convert the Mac version.

iPad drag and drop in action.

iOS 11’s drag and drop framework has been written from scratch to be optimized for multitouch input, multitasking between apps, privacy and security, and a flexible API. While the underlying motivation might have been the same that drove the Macintosh team to ship drag and drop, the implementation in iOS 11 is a completely new effort from Apple – the result of years of testing and perfecting a new API for developers. Put another way: iOS drag and drop sounds similar to the same feature on the Mac, and it shares some common traits with it, but under the hood it’s been reimagined for the age of apps.

In this landscape, drag and drop coexists with other initiatives that Apple promoted to facilitate the exchange of content between apps, such as extensions, document pickers, or the pasteboard. Those features are still with us in iOS 11, but the need to rely on them as inter-app communication mechanisms is being largely obviated by drag and drop. This will cause a massive shift of iPad workflows from modular to integrated; drag and drop results in a continuous transfer of content that is faster and more natural than extensions ever were.

As we’ve seen with the dock and Split View, drag and drop is also a new beginning for iOS multitasking. The dock’s integration with Split View and Slide Over is made possible by drag and drop; without it, the dock would lose most of its utility, and we’d probably still be stuck with the same limitations of iOS 9.

For third-party developers, drag and drop presents an opportunity to rethink how much their apps can dynamically communicate with others. While the drag and drop API is easy to get started with, it gets surprisingly deep with multiple representations of dragged items, custom previews, and dedicated animations for drop targets. The feature could unexpectedly break in a variety of ways. Developers will have to put in the work to properly support drag and drop to the extent pro iPad users expect.

The upside, though, is enormous: drag and drop simplifies notoriously tricky interactions for third-party apps on iOS – such as importing data in a specific format, or exporting multiple items to another app – while creating an ecosystem of apps that now speak the same language. Apps without drag and drop will soon stick out as much as apps without Split View two years ago.

The impact of drag and drop on the iPad experience is so profound, it’s no exaggeration to consider it the next big step for multitouch. Drag and drop defines iOS 11, and in the process, it reinvents iOS for iPad altogether.

The Flow of Drag and Drop

In broad terms, drag and drop consists of picking up content from a source and dropping it in a destination. The source and the destination can be the same app, but, on iPad, a drag activity can start in one app and end in another.

A drag always begins by long-pressing on selectable content until it lifts up; at that point, while still touching the screen, you can start dragging the item away and navigate the iOS interface to find a destination.

iOS drag and drop doesn’t move content from the source to the destination: a copy of the item is created by default, with some exceptions. When moving an item between two different apps – such as a text selection from Notes dropped into a Mail message – the item is copied into the destination. Conversely, if the drag activity occurs within the same container – such as dragging a paragraph of text to a different part of the same note’s body – the item is moved.

The general rule of thumb is that drag and drop can’t be disruptive when the activity involves different apps, but content shouldn’t be duplicated when inside the same app whenever possible. When the source and the destination are the same, developers must decide the best way to handle the drag activity to match user expectations.

Drag and drop is powered by an API that provides apps with multiple layers of control for dragging items and receiving them. The feature is supported out of the box for standard text and web views, even in apps that haven’t been explicitly updated for iOS 11 yet. As an example, over the summer I could drop text into the Slack compose field or drag links out of Editorial’s built-in browser while both apps were still compiled for iOS 10. By giving drag and drop “for free” to UITextView, UITextField, and WKWebView, Apple has ensured that basic support for drag and drop can be available in standard text fields and web views of every third-party app at launch.

Editorial and Slack are examples of apps that supported some drag & drop actions out of the box.

As a major addition to iOS inter-app communication, Apple expects drag and drop to be pervasive and embedded in thousands of apps. As such, the company’s guideline is that all selectable content should support drag operations, while all editable content areas should expect other items to drop in at any moment. To achieve consistency and simplicity in the experience, there are several recurring elements for apps adopting drag and drop.

Both the drag and drop activities can have a simple, almost default appearance (a translucent representation of the content being dragged⁶⁰), or take on more sophisticated and custom behaviors.

Examples of drag item previews.

Developers can customize both ends of the session: they can tweak the lift animation, what types of items can be dragged, and how the preview of each drag item should be rendered; and, they can define the appearance and affordances of the destination when a drag item is placed over the target, as well as the drop animation when items are set down. These APIs are unique to drag and drop on iOS, and they’ve been designed to give developers greater control over the touch interface and different metaphors of iOS compared to macOS. Robust drag and drop implementations will stand out thanks to the richness of drag item previews and use of animations to complement multitouch.

Most notably though, iOS 11’s drag and drop is characterized by another fundamental difference with the Mac: with multitouch, drag and drop on iOS supports non-contiguous multi-select to drag a collection of items from a source (or even multiple sources) to the destination.

There are two versions of multi-item drag and drop: similar items from the same app can be added to an in-progress drag session while a single finger is holding a stack of items, or you can use multiple fingers to create simultaneous drag sessions, each individually managed by the finger holding an item.

Adding multiple items to an in-progress drag requires apps to specify if this mode is supported, and how or where it should be activated. In most cases, you will be able to flock an item over to an in-progress drag by holding the first drag item with one finger and tapping other items in the app to select them. In Photos, for instance, you can hold a photo to begin a drag session, then tap other images to add them to the stack.

Additionally, developers can customize drag and drop so that only items of the same type can be added to a drag. In Mail, you can create a stack of email messages, but you can’t select an attachment and add it to the same drag session.

In that case, you’ll have to take advantage of multiple fingers to create concurrent drag sessions where each drag item (or stack of items) can be dropped in a destination at separate times. This contortionist-like maneuver isn’t the fastest or most accessible one to pull off, but it’s technically impressive when you consider how, in theory, you can create up to 10 individual drag sessions and hold onto them with all your fingers.

In practice, this can be a nice way to collect different bits of content from a variety of apps and drop them in succession somewhere else, like text from Notes and screenshots from Photos dropped into the same email message in Mail’s composer. Multi-session drag and drop even supports transferring drags between fingers by passing an item to another finger mid-flight. It seems crazy, but it works – all while constantly hitting 60 frames per second and adhering to the user’s fingers.

Real-time reordering of items in the destination also plays an essential role in communicating the effect a drag item will have on an app’s existing content. For collection and table views, Apple added an API to generate a Home screen-like reordering of items that creates gaps between them while you’re hovering with a drag item over the view.

The reordering animation helps you understand where drag items will end up once dropped; Apple went as far as adding three levels of cadence (fast, immediate, or slow) that developers can use when reordering items.

In the lifetime of a drag, what happens once a drag is in progress but just before the user commits to the drop will push developers to think deeply about interface affordances, indicators, and other visual cues for potential destinations in their apps. Apps should use a combination of color highlights, animations, badges, and other custom UI elements to inform users whether a drag item can be accepted or not without forcing them to try and see what happens.

There are some good examples of these practices in Apple’s apps. Mail and Photos use gray cell highlights to indicate you can drop messages into mailboxes and photos into shared albums, respectively. In general, any table view can highlight drop destinations through colors; in Notes, folders that can receive dragged items are highlighted in gray, while notes that can receive text or media are highlighted in yellow in the main list.

Dropping a note in a different folder.

Notes (unsurprisingly, the app with the richest support for drag and drop in iOS 11) also uses the yellow blinking cursor to tell you, visually, where content will be dropped within a note while you’re still holding an item and dragging it across text.

Dropping text in Notes.

Badges can be attached to an in-progress drag to hint at supported actions or forbidden operations. Dragging a link from a webpage in Safari reveals a green + badge when the link is floating towards the address bar to suggest it’ll be opened in a tab. On the other hand, dragging a link and hovering over a folder in the Notes sidebar will display a gray ⃠ badge, which means the item won’t be accepted by the destination, resulting in a cancelled drag session. These symbols (and others, including numeric badges) can be used by third-party developers in their apps.

Different indicators for drag items.

Last, spring-loading. Like on macOS, you can drag an item over a destination and pause briefly to open it directly. Spring-loading is more relevant on the Mac, which doesn’t have a secondary cursor to click on destinations, but iOS 11 apps can also implement it for some of their controls. You can spring-load a tab in Safari by holding a drag item over it; in Reminders, you can organize tasks in lists by spring-loading them.

Multitouch makes spring-loading close to irrelevant on iOS, and Apple is advising developers to use it as an embellishment of the interface limited to a subset of controls – never as the primary way of interacting with drag and drop. Still, it’s another option developers have, which could also help users who don’t want to use multiple fingers for dragging items and choosing destinations.

The expectation that all selectable and editable content should support drag and drop will lead users to be disappointed when an app doesn’t respond to these new modes.

Apple has the clear goal of ensuring that all apps can integrate with drag and drop on iOS 11. To set an example for developers, they have meticulously updated most of their apps with different drag and drop interactions and designs – a sign that this functionality has been in the works for a couple of years at least. The company’s built-in implementations are solid; the API is easy to understand, yet customizable enough for developers to devise clever ways to enrich their apps with drag and drop too.

The flip side of this, however, is that lots of trial and error will be initially involved with figuring out where and how drag and drop is available. And because drag and drop exists in multiple forms across a variety of apps, it’s important to understand how everything works behind the scenes.