Going Spatial: What Industry Leaders Need to Know About Apple Vision Pro and WWDC '23

Day one of Apple’s annual Worldwide Developers Conference (WWDC) first unveiled incremental advancements across Mac hardware, iOS, iPadOS, MacOS, WatchOS, and connected home devices, many of which will have a considerable near-term impact.

But the major announcement — and unanswered question — from this year's WWDC was undoubtedly Apple Vision Pro, offering a tantalizing glimpse into one potential future of interactive technology. Jettisoning the typical “AR/VR/XR headset” vocabulary in favor of the category-creating term “spatial computing,” Apple Vision Pro may transform how we interact with our work, our media, and even our personal and institutional memories.

Any new tech requires thoughtful scrutiny and patience before we can guess at long-term implications, and the initial use cases and adoption rate are not yet clear. That said, as strategists, developers, and designers, we’re excited to explore the new platform.

“Apple emphasized the importance for developers to create experiences for the Vision Pro, asserting that the device will transform every industry if executed correctly,” reports WillowTree iOS Engineer Marianna Mikhael, on-site for this year’s WWDC. “Apple aims to make every existing mobile and web app available on Vision Pro — unless brands explicitly opt out — and they’re launching a completely separate app store for the platform. It will be crucial to ensure mobile and web apps are compatible with this new paradigm.”

"We can't wait to see what developers will do with this groundbreaking platform," said Susan Prescott, Apple's VP of Worldwide Developer Relations.

With this in mind, the following questions require the most immediate consideration from industry leaders looking to get ahead of these developments:

Long-Term

• While “Pro” is typically a moniker for Apple's highest-end, most performant devices, with price tags to match ($3499), will there be demand for the device and widespread adoption in its current incarnation?
• Will enterprises and individuals adopt this technology (and how quickly) to the point where it becomes a primary computing device versus a peripheral?
• Do our answers to the questions above meaningfully change if I envision a more streamlined future-state version of the device at a lower price point?

Short-Term

• What is our initial Vision Pro compatibility strategy? How quickly can we migrate and test existing apps to ensure functionality and parity on this new platform?
• Which of our current employee and customer experiences would benefit from the efficiencies of a multimodal interface driven by voice, hand gestures, or eye movements coupled with an immediate screen-based response?
• How can we improve or develop features and experiences that leverage spatial computing capabilities to delight the next generation of users?

Below, we’ll explore the details of this new platform and suggest initial use cases (encompassing the Vision Pro and “smaller” WWDC announcements across other platforms) that stand to impact Healthcare & Wellness, Travel & Hospitality, Media Delivery, Consumer Goods & Retail, and the increasingly digital workplace.

Let’s get into it…

‍

Multimodal User Interface

Apple has long been a pioneer in intuitive input models, consistently pushing the boundaries of human-device interaction. Continuing this trend, the Apple Vision Pro redefines the user interface paradigm with its 3D spatial computing interface.

Users manipulating windows and apps in a three-dimensional space without additional peripherals is a significant departure from the AR/VR headsets on the market, which typically require additional handheld devices and cut off users from their real-world surroundings. Instead, Apple Vision Pro's "EyeSight" technology allows users to interact with the virtual and real-world simultaneously and more naturally using eye movements, hand gestures, and voice controls.

WillowTree has long been bullish on these types of user interfaces — especially multimodal voice — which are more human and more efficient. Vocable — a free, open-source augmentative and alternative communication (AAC) mobile platform we developed — allows users to communicate via eye movements, for instance, and is a natural fit for this new platform.

"Every major Apple platform was driven by an innovative new input model,” said Apple VP of Human Interface Design Alan Dye. “Mac, with the mouse. iPod, with the click wheel. And iPhone, with multitouch. With Vision Pro, we set the ambitious goal to design an incredibly intuitive input model for spatial computing — one that could be used without controllers or additional hardware. Apple Vision Pro relies solely on your eyes, hands, and voice. It's just you and your content. It's remarkable and it feels like magic."

Our Take: Done right, we agree that the UI capabilities of the Vision Pro signal a potentially significant step forward in multimodal user experience. However, any new input method requires a period of transition, so adoption will depend on how intuitive Vision Pro is for users.

‍

​​Transforming Work and Play

Apple Vision Pro could be a game-changer for various sectors, especially Entertainment and Media Delivery. But it also impacts nearly every industry’s increasingly hybrid virtual-physical workplace.

The headset's expansive screen allows for an "infinite canvas." From creative users who require more visual real estate to financial services professionals who appreciate that 'war room' vibe of multiple dashboards open simultaneously to any field technician who’d benefit from a more “heads-up, hands-free” experience, the device may revolutionize workflows. Moreover, if multiple people enter that same immersive, infinite canvas, the remote collaboration capabilities of Apple Vision Pro are seemingly unparalleled.

The implications for media consumption are equally profound. Apple Vision Pro provides a more immersive media delivery experience with its "Spatial Photos" and "Spatial Video" features. These allow users to capture and relive memories in full 3D, placing you directly into the action of images, panoramas, movies, and games. “Spatial” is not simply a new screen format or technical frontier; it may dramatically reshape the entertainment industry’s approach to content creation and creativity.

Our Take: The key questions will be whether the device’s size and cost generate widespread adoption in the workplace and home; and whether these experiences truly encourage human collaboration and connection versus more isolation. For instance, can the communal experience of “gathering around the TV” really be supplanted by multiple individual devices? We're skeptical.

Given the inclusion of “Pro” in the name, Apple likely knows that this is a price point and adoption curve initially aimed at enterprise use cases around workplace productivity, along with some “toe in the water” concepts from media entertainment companies (as evidenced by the Disney tie-in at WWDC).

‍

Advanced Hardware and Customization

To ensure user comfort and compatibility, Apple Vision Pro is designed with a removable strap and will be uniquely fitted to individual users. However, the headset's cutting-edge sensors, cameras, and under-the-hood tech truly set it apart (the device purportedly encompasses some five thousand patents).

With LIDAR, downward cameras, IR cameras, LED illuminators, and a host of audio advancements, Apple Vision Pro’s suite of high-tech hardware may redefine user experience. In addition, it employs Apple's R1 processor, which aims to eliminate the lag and latency often associated with motion sickness during headset use, a significant barrier to entry across the existing AR/VR/XR market.

Our Take: We may have to guess until demos are widely available. Given Apple’s track record, we’re confident the quality and fidelity of the Vision Pro experience will be impressive. Still, the device's size, weight, and battery life will likely remain a factor through early iterations of this technology, as will user reports of nausea and disorientation.

‍

VisionOS: Creating Your Digital Persona

One of the most innovative and likely polarizing aspects of visionOS will be the digital persona created for each user. This next-level avatar system acts as a deepfake stand-in during video conferencing so teams aren’t staring at a room full of goggled colleagues.

Our Take: This may seem unnerving, as it generates a highly-realistic Memoji that might induce an 'uncanny valley' effect. However, as we become more accustomed to these digital representations, they could become an accepted norm in our increasingly virtual world. Our user research team will actively explore this space in the months ahead.

Future-Proof Privacy and Security

To ensure security and privacy, Apple is implementing OpticID, a built-in retina scanner. While the device does track eye movement, Apple asserts that only data equivalent to 'clicks' are captured, not the second-to-second direction of your gaze.

Our Take: This commitment to privacy (and Apple’s ability to make good on it) could have significant implications for user trust and data security. Known and unknown security issues accompany any new tech, and success will depend on Apple’s and the developer community’s ability to stay ahead of the game.

‍

New Use Cases: Apple Vision Pro + Other WWDC Announcements

Apple demonstrated a range of compelling use cases for Vision Pro. Combined with key announcements across Apple’s other devices and operating systems (iOS, WatchOS, etc.), here’s our take on what executives need to prepare for in various industries:

‍

1. Healthcare & Wellness

• WatchOS 10's State of Mind feature: This digital experience allows users to log momentary emotions, which can be used alongside other Health data to paint a more three-dimensional portrait of wellness and help determine risk factors for increasingly common issues of depression and anxiety. If this data-driven approach to mental health can be integrated into new and existing health and wellness experiences, it’s another potential game-changer.
• iPad HealthKit updates: Enhancements allow for more on-screen data, which could benefit healthcare professionals requiring a high volume of real-time data on a single screen.
• Apple Vision Pro for Healthcare: 3D models and environments can be used for highly immersive and realistic education and training experiences in medical and healthcare fields.

2. Travel & Hospitality

• Inline Location and CheckIn features in iOS: These new features allow for real-time user-to-user location sharing, which may have applications in the travel and hospitality industries, improving everything from customer experiences to logistics tracking.
• Journal in iOS: This AI-driven feature curates memories and experiences — bringing together photos, location data, music, and other elements into an easy, elegant format for reflection and sharing — a tool tailor-made for travel.
• Offline Maps in iOS: a long-awaited feature that may prove invaluable for overseas travelers or those in areas with poor network coverage.
• Apple Vision Pro for Travel: the immersion offered by Apple’s new device may be a double-edged sword, both shrinking physical distance to make travel less essential and inspiring new possibilities for unfamiliar experiences.

3. Media Delivery

• Apple Vision Pro for Media: This device could revolutionize the media industry, offering immersive and interactive viewing experiences. It can be used to create panoramic images and videos, and the Spatial Photos and Spatial Video features allow for fully immersive 3D experiences.
• New tvOS and AppleTV integrations: On a more familiar scale, it’s now easier to port a FaceTime call or blockbuster film to a TV, effectively expanding screen real estate for better interpersonal communication or media consumption.

‍

4. Consumer Goods and Retail

• NameDrop and AirDrop enhancements in iOS: These features could streamline sharing information, impacting retail experiences by allowing consumers to quickly and easily share product information.
• Apple Vision Pro in Manufacturing: Showing 3D renderings of production options or assembly line operations could provide a more interactive shopping experience, especially for more bespoke products that are customized or assembled to order.

Despite all of this speculation, the bottom line is: platform compatibility.

Those who adopt this new device and set of experiences will require their favorite, familiar heavy-use digital experiences to migrate to the Vision Pro along with them. Put simply, if an app is available on the web, mobile, and tablet devices, users will increasingly expect it to also be available on the Vision Pro platform and other similar immersive devices, whether or not the functionality is at all transformed.

The Apple Vision Pro and other updates announced at this year's WWDC may signal a paradigm shift towards more intuitive, immersive, and interconnected digital experiences. As developers, we're excited to explore these new horizons and help shape the future of digital interaction.