App Development

iOS 9 Picture in Picture: 9 Things You Need To Know

picture-in-picture blog-featured-image tc-510x295

iOS 9 finally brought what everyone has been waiting for, multitasking. The Slide Over, Split View and Picture in Picture (PiP) have made the iPad a more powerful and handy tool than ever before. Having everything you need on the screen at the same time increases productivity at work, which will definitely be appreciated in the business world. Instead of switching back and forth between apps, the user can focus on things that are really important. And now, thanks to Apple’s elegant solution for video-streaming apps, you don’t have to pause your favorite TV series while replying to a friend’s email, or while you browse or search for other episodes. Apple’s also made it easy for us to look things up on the Internet while we’re talking to someone on FaceTime with iOS 9.

All these multitasking features are really great, but it’s the Picture in Picture (PiP) feature that makes the iPad stand out from the rest of Apple’s products. Even on a Mac, playing a video in the corner while having a few other windows open can be challenging as you have to worry about things like resizing the window properly, the video player being covered by other windows, etc. In my opinion, PiP would be a nice add-on to OS X, but we’ll have to leave that for Apple to decide.

If you are still asking yourself if you really need to support this feature in your iOS app, the answer is yes, absolutely! It’s a feature that’s going to be in great demand for every video streaming app,  and you are sure to disappoint your target audiences without it. Besides, including it in your app won’t require much effort from you, as you’ll find out further below.

Picture in Picture Q & A

1. How does it work?

Once you turn the PiP mode on by tapping the PiP icon on your video-player toolbar, the video will be displayed in a floating window which will stay on top of other apps opened by the user.

The PiP window can be resized and dragged to all four corners of the screen - it will snap in place, it can be moved off the screen as well. Tapping the window will show the controls, which allow you to play/pause the video, quit PiP mode, and scroll through the video.

2. What happens if the user plays another video (or audio) while my app is running in PiP mode?

No need to worry about that. Your PIP will be automatically paused, and if the user wants to replay your video, then the other video/audio will be paused.

3. Can the user disable the invocation of Picture in Picture mode?

Yes, and it can be done in Settings > General > Multitasking > Persistent Video Overlay. So make sure this option is enabled while testing!

4. How do I support Picture in Picture in my app?

It’s fairly simple. If you play your video content using the AVPlayerViewController class, the PiP button is already there and the system will handle the switching between PiP and full-screen mode. If your video content is played in WKWebView, it can also be displayed in PiP mode. However, if you want to employ a self-made custom video player, it will require a little more work. You would have to use an instance of AVPictureInPictureController class from AVKit, which you would associate with an AVPlayerLayer instance. In either case, the video playback in your app has to be made eligible for PiP.

  1. In the Build settings of your project, set the Base SDK to “Latest iOS.”
  2. Go to your project’s Target. In the Capabilities view tab, select Background Modes >  Audio and AirPlay.
  3. Your app’s audio session has to be set to AVAudioSessionCategoryPlayback or AVAudioSessionCategoryPlayAndRecord.

For example, in WWDC demo source code it is done in AppDelegate.swift:

audioSession = AVAudioSession.sharedInstance()


do {


try audioSession.setCategory(AVAudioSessionCategoryPlayback)


}


catch {


print("Audio session setCategory failed")


}

*Note that PiP mode will be available when your video is playing fullscreen.

5. How do I handle the transitioning to PiP mode if I am using a custom video player?

The very first step is to ensure that your app is eligible for PiP mode (see above) and that the video content can be displayed full screen.

Since you are not using AVPlayerViewController, you will need to create a button to switch to the PiP mode. However, you have to check first that this mode is supported on the user’s iPad. AVPictureInPictureController class serves that purpose. So initially your app will only be displaying the PiP control only if AVPictureInPictureController.isPictureInPictureSupported() method returns true.

AVPictureInPictureController is what actually can initiate and quit the PiP mode via methods:

startPictureInPicture()
startPictureInPicture()

It can give you the information about the current PiP states, such as:

pictureInPicturePossible - tells you if the PiP window is available to use. If another app is using the PiP mode, it returns false. This property is available for key-value observing, which makes it easy to hide or disable the PiP button whenever this state changes.

pictureInPictureActive - shows whether the PiP window is on the screen at the moment.

pictureInPictureSuspended - returns true if the video playback in your app is paused and not visible because another app is using PiP. Your video will be resumed when another app quits  PiP.

The AVPictureInPictureController is initialized with the AVPlayerLayer of the view which will be displaying the video. For that purpose, the video player view was set up to return its layer as AVPlayerLayer:

class PlayerView: UIView {


var player: AVPlayer? {


get {


return playerLayer.player


}


set {


playerLayer.player = newValue


}


}


var playerLayer: AVPlayerLayer {


return layer as! AVPlayerLayer


}


override class func layerClass() -> AnyClass {


return AVPlayerLayer.self


}


}

This is how the PiP setup is done in WWDC source code once the video content is ready to be displayed:

private func setupPictureInPicturePlayback() {


		if AVPictureInPictureController.isPictureInPictureSupported() {


			pictureInPictureController = AVPictureInPictureController(playerLayer: playerView.playerLayer)
			pictureInPictureController.delegate = self
			
			
			addObserver(self, forKeyPath: "pictureInPictureController.pictureInPicturePossible", options: [.New, .Initial], context: &playerViewControllerKVOContext)
		}
		else {
			pictureInPictureButton.enabled = false
		}
}

Note that in this example, the view controller containing the video player conforms to AVPictureInPictureControllerDelegate protocol. It defines methods which let you respond to the PiP playback events. Those include:

pictureInPictureController(_:restoreUserInterfaceForPictureInPictureStopWithCompletionHandler:)
pictureInPictureControllerDidStartPictureInPicture(_:)
pictureInPictureControllerDidStopPictureInPicture(_:)
pictureInPictureControllerWillStartPictureInPicture(_:)
pictureInPictureControllerWillStopPictureInPicture(_:)
pictureInPictureControllerFailedToStartPictureInPicture(_:withError:)

The following code disables the PiP button according to the value of pictureInPicturePossible property:

override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [NSObject : AnyObject]?, context: UnsafeMutablePointer<Void>) {


…


            if keyPath == "pictureInPictureController.pictureInPicturePossible" {
			let newValue = change?[NSKeyValueChangeNewKey] as! NSNumber
			let isPictureInPicturePossible: Bool = newValue.boolValue
			pictureInPictureButton.enabled = isPictureInPicturePossible
		}
}

Finally, if the user taps the PiP button, the AVPictureInPictureController would have to initiate or stop the PiP mode:

@IBAction func togglePictureInPictureMode(sender: UIButton) {
		if pictureInPictureController.pictureInPictureActive {
			pictureInPictureController.stopPictureInPicture()
		}
		else {
			pictureInPictureController.startPictureInPicture()
		}
}

Simple, as promised!

6. Can I opt out for PiP for a particular video?

Yes! All you need to do is set the allowsPictureInPictureMediaPlayback property of the AVPlayerViewController or WKWebView instance to NO.

7. Can I subclass AVPictureInPictureController?

Even though there is a big temptation to subclass AVPictureInPictureController, doing this is not recommended. Apple warns you that “overriding this class’s methods is unsupported and results in undefined behavior.” Also, it might prevent your app from being approved by the App Store.

8. Is there a way to apply  a theme to the PiP window, change the standard controls, add more controls, …?

For now, the official AVKit framework documentation does not mention any PiP user interface properties that are accessible to developers. Yes, this is one slightly disappointing thing about it, let’s hope that Apple will give us more freedom with the next version of iOS.

9. Any best practices regarding the usage of Picture In Picture?

Basically, the same practices that are applied to background processing. You have to be mindful of  consuming too many resources in the background, which creates a risk of your app being terminated by the system, so it is necessary to free the unused resources whenever possible once your app transitions into PiP mode.

Quickstart-Guide-to-Kotlin-Multiplatform

A Quick Start Guide to Kotlin Multiplatform

Kotlin Multiplatform, though still experimental, is a great up-and-coming solution...

Read the article