App Development

Drag and Drop in iOS 11

iPhone 7 sitting atop a wireless Mac keyboard.

At WWDC this year, Apple unveiled an exciting new feature for iOS 11: drag and drop. The basic idea is familiar to anyone who has used a desktop computer before: drag something from one place on the screen and drop it somewhere else. This seemingly simple interaction underpins much of how users move data on desktop computers. Users drag and drop text between documents, copy images from websites into photo apps, sequence video clips in video editors, and move files between browser windows using this natural UI idiom.

Apple has adapted this feature for mobile devices and, in doing so, given app makers the ability to enable powerful new workflows for users. This is especially true on the iPad, where drag and drop works hand-in-hand with the platform’s ever expanding multitasking capabilities. Previous versions of iOS let users view and work with two apps simultaneously on the iPad. With the addition of the Drag and Drop feature on the iPad, users can now move data between apps in a way they’ve never been able to do before.

But what kind of data can they move between apps? That’s up to us, the app makers. Apple has given us the ability to define when and what data can be exported or imported through Drag and Drop. However, users are going to expect to be able to move their app’s data in the way that supports their workflow. App makers can no longer expect their apps to work in a vacuum. They must open up to providing and receiving data in order to participate in new user-defined multitasking workflows on iOS.

Drag and Drop Features

At WWDC this year, Apple said they had three main goals with Drag and Drop. First, they wanted it to be responsive. To that end, they’ve made it activate and update quickly in response to user input. Next, they wanted it to be secure, which they’ve done by giving app makers a secure framework through which we can control if and how data is imported or exported. Finally, they wanted it to be “a great multi-touch experience”. They’ve accomplished this last goal by entirely rethinking Drag and Drop for mobile. Use this feature, and it’s immediately apparent that this is no simple port of desktop drag and drop interaction. On a desktop machine, the single mouse pointer is given over entirely to a single drag and drop action. In iOS 11, we can use simultaneous touch gestures to tap and drag multiple items, add to “flocks” of dragged item groups, navigate the OS, and even open and use new apps, all while maintaining our dragged items.

Single App Features

Drag and Drop is not just a new way to do app multitasking. It’s a great feature to use within a single app as well. Of note is that Apple has built extensions to Drag and Drop that specifically support table views and collection views. This means we could use Drag and Drop to implement item reordering, insertion, and replacement on table and collection views in our apps. Until now, we’ve relied on using an “edit mode” with an alternate interaction model, but we shouldn’t need that once we can implement Drag and Drop on these views. Table and collection view Drag and Drop is an awesome capability that could benefit many iOS apps, not just iPad ones. While the iPhone supports all Drag and Drop APIs, Apple has made the decision that Drag and Drop will operate only within the same app on that platform. This has the interesting implication that, for a “universal” app able to run on both iPhone and iPad, we can make available expanded Drag and Drop capabilities when running on the iPad.

Basic Multi-app Features

At the very least, users will expect to be able to use Drag and Drop to do what they do now with Cut and Paste: move or copy text and images between apps. We’ve done this for nearly a decade on mobile, albeit with limited platform support. With Drag and Drop, any view in our application is a potential Drag data provider. To enable this functionality, we need only add a drag interaction to the view and implement the required delegate method.

view.addInteraction(UIDragInteraction(delegate: self))
 
func dragInteraction(_ interaction: UIDragInteraction, itemsForBeginning session: UIDragSession) -> [UIDragItem] {
    // Return an array of items containing data to be dragged
}

Similarly, we can make any view a Drop receiver by adding a drop interaction and implementing its delegate methods.

view.addInteraction(UIDropInteraction(delegate: self))
 
func dropInteraction(_ interaction: UIDropInteraction, canHandle session: UIDropSession) -> Bool {
    // Return true if the app can accept the drag data in session
}
 
func dropInteraction(_ interaction: UIDropInteraction, sessionDidUpdate session: UIDropSession) -> UIDropProposal {
    // Return a proposal indicating the operation type (move, copy, etc.)
}
 
func dropInteraction(_ interaction: UIDropInteraction, performDrop session: UIDropSession) {
    // Get the item provider from the session, load the data from the item provider, and update the app
}

That’s all we need to get started dragging and dropping! But Apple has provided us so much more. Nearly every bit of the Drag and Drop experience is customizable.

We can:

  • Customize the lift, cancel, and drop animations
  • Make custom views to visually represent our drag items as the user moves them
  • Make springload interactions for views when a dragged item passes over those views
  • Animate the rest of our app alongside the lift and drop

Most of this is available through the Drag and Drop interaction delegate interfaces, so it’s easy to experiment with the look and feel of our Drag and Drop interactions.

Advanced Multi-app Drag and Drop

Suppose that we don’t just want to drag and drop text and images. Our app has custom data types that we’d like to drag and drop internally or within a group of our apps. We’d also like to accept file data that originates entirely from outside our apps. This is all possible through new capabilities on the NSItemProvider class.

Let’s back up for a second and discuss exactly what data is passed around during a Drag and Drop. In the data provider’s UIDragInteractionDelegate.dragInteraction(_ interaction:, itemsForBeginning session:) delegate method mentioned above, we return an array of UIDragItem objects. Each UIDragItem contains an NSItemProvider, which wraps the data we ultimately want to send. Similarly, in the data receiver’s UIDragInteractionDelegate.dropInteraction(_ interaction:, performDrop session:) delegate method mentioned above, we receive a UIDropSession containing the array of UIDragItem objects, each of which contains an NSItemProvider that wraps the data we want to receive. The NSItemProvider embodies one or more “representations” of the data item being transferred. For example, if we were dragging an image, the drag provider could supply an NSItemProvider that represents the image as both a HEIF and a JPEG. If we then drop the image on a receiver able to handle only JPEG and PNG images, the receiver would be able to choose to load the JPEG representation from the NSItemProvider. In this way, the NSItemProvider helps us negotiate the format of the data passed between provider and receiver.

So how do we enable Drag and Drop on our custom class? We make it conform to the NSItemProviderReading and NSItemProviderWriting protocols, which respectively provide data serialization and deserialization functionality when implemented. Now we can initialize an NSItemProvider with an instance of our custom class upon Drag, and we can extract an instance of our custom class from the NSItemProvider on Drop. In each of our NSItemProviderReading and NSItemProviderWriting protocol implementations, we provide an ordered list of preferred data formats and a data serialization/deserialization method that operates based on a requested data format. The data format is specified using Apple’s Uniform Type Indicators (UTI), and it’s automatically decided upon by the NSItemProvider, which chooses the most preferred common data format between sender and receiver.

If we want to Drag and Drop a file between apps, the NSItemProvider class works in a similar way. This time, however, we don’t implement serialization protocols on a custom class. Instead, when the user drags a file from within our data provider app, we register with the item provider a “representation” of our file. We provide this representation with a UTI (standard or custom) and a file serialization function. When the user drops a file within our data receiver app, we load the file from the item provider’s “representation” using a deserialization function we define.

Don’t worry if this all sounds a bit complicated. There are easy ways to start building Drag and Drop into your app that will provide immediate benefit to your users. So where should you start?

Receiving

We recommend the approach to adding Drag and Drop support that Apple suggested at WWDC: start by receiving Drops. It’s easy to enable dragging and dropping text into our apps’ text fields. In fact, UIKit’s built-in text fields may enable Drag and Drop by default when iOS 11 ships. However, we shouldn’t stop there. Remember, any view is a potential Drop receiver. Suppose we had an app with a list of customers’ contact information. Right now, to add a new contact, we would probably have to tap a “New Contact” button and fill out form full of text fields. Wouldn’t it be cool to create a new customer contact in our app by simply dragging contact information from, say, an email and dropping it directly into the app’s contact list? We’ll be able to implement that Drop feature easily because we can design where our app receives the drop and how it interprets the text data.

We’ll be able to create the same kind of functionality with images as well. In iOS 11, many core types like UIImage will already conform to the item provider protocols that enable automatic loading of their data upon receipt, so building out great Drop features for our users should be a snap.

Providing

Next, we should think about how to provide Drag data in our app. Because of the special Drag and Drop extensions Apple is building into table and collection views, these views make great places to experiment with adding Drag functionality. We can do this without worrying too much about the data representations we make available since we’ll usually be dropping the data within our own app. iOS 11 makes it easy to implement cell removal, insertion, and replacement using Drag and Drop. It even creates and manages placeholder cells, which are displayed while data loads after a drop.

Next, we should think about how to use Drag to provide data to other apps. This is more difficult to answer because it requires us to think differently about our app’s data. We must consider how the user would be able to use our app’s data outside of the app. In fact, from the user’s perspective, much of our app’s data is actually their data, which they’re entitled some control over. Where possible, we should empower users to work with their data by enabling Drag and Drop. We should think about what data we can export as files to the new Files app in iOS 11 or to other apps entirely. Opening up our app’s data will enable users to put together multi-app workflows that we hadn’t even considered.

Challenges

While we believe Drag and Drop to be a real game changer for the iPad as a pro computing platform, some challenges lie ahead.

Security

Apple has done a good job of putting power in the hands of app makers to limit export of app data. We can choose to only provide the data we want through Drag and Drop, and we can choose to share that data only within our app, within our app group, or with all apps on the user’s device. Likewise, we can have our app receive only the types of data we want, and only from the sources we approve. However, it’s up to us to perform our due diligence and only open up app data to Drag and Drop if it’s safe to do so for our users.

Discovery

Drag and Drop is entirely new to iOS. If we build these new Drag and Drop features into our apps, how do we help the user discover them? We hope Apple will help out with user education around Drag and Drop through their iOS 11 media campaign. They’ve done similar user education when rolling out important features in the past. Once users learn about Drag and Drop as a concept in iOS, they should be able to quickly embrace dragging and dropping text and images. How do we indicate Drag capability for other kinds of data? Perhaps more challenging, how do we indicate where our apps accept Drops and what kinds of data those Drops can contain? iOS 11 provides some technical solutions to these challenges. The drop delegate “proposal” system allows a dragged item to display an icon indicating if it can be dropped when hovering over a potential receiver. Further, we can allow our app to accept multiple data formats, which is good for negotiating a “least common denominator” data type between a Drag data provider and a Drop data receiver. As such, I believe we app makers must be good citizens in providing and accepting data in the common formats of our apps’ work domains. For photo apps, this could mean providing and accepting multiple image formats such as PNG, JPEG and HEIF. For word processing apps, this might mean accepting LaTeX, PDF, MSWord files as well as HTML, RTF and plain text snippets. We will also need to rely on our designers to come up with design language around Drag and Drop that makes sense to our users as well as design cues from Apple itself.

Resources

For more information about Drag and Drop in iOS 11, I highly recommend checking out the WWDC 2017 session videos as well as their accompanying Xcode 9 Beta sample projects.

Introducing Drag and Drop

Mastering Drag and Drop

Drag and Drop with Collection and Table View

Data Delivery with Drag and Drop

Download Xcode 9 Beta

Quickstart-Guide-to-Kotlin-Multiplatform

A Quick Start Guide to Kotlin Multiplatform

Kotlin Multiplatform, though still experimental, is a great up-and-coming solution...

Read the article