One of the best indicators of how sticky a cross-device utility will be in my setup is how quickly I install it everywhere. For Sindre Sorhus’ Scratchpad, the answer was “very sticky.” The simple text utility works on the iPhone, iPad, Mac, Apple Watch, and Vision Pro (via iPad compatibility mode), and as soon as I tried it on my desktop Mac, I grabbed all of my other devices and installed it on them, too.
At its core, Scratchpad is a single view for typing or pasting plain text that syncs everywhere. What sets it apart from similar apps is its many small touches that demonstrate a deep understanding of the way people use a scratchpad app.
Apple revealed a new Immersive Video title for the Vision Pro. As announced at SXSW today, Vision Pro users will be treated to a live performance of three Metallica songs: “Whiplash,” “One,” and “Enter Sandman” on March 14th.
According to Metallica’s press release:
This project marks a new foray into immersive technology, using ultra-high-resolution 180-degree video and Spatial Audio to give fans unprecedented access from vantage points as close up as the Snake Pit to wide-angle views. It brings the live show to a whole new level, and to achieve this, Apple built a custom stage plot featuring 14 Apple Immersive Video cameras using a mix of stabilized cameras, cable-suspended cameras, and remote-controlled camera dolly systems that moved around the stage.
For its part, Apple released a trailer for the video on YouTube:
along with an interview by Zane Lowe with Metallica’s Lars Ulrich:
Today’s Metallica news follows the recent Immersive Video announcements of VIP: Yankee Stadium and Bono: Stories of Surrender. It’s great to see new content coming to the Vision Pro, especially live concerts and sports, which are a perfect matches for the format.
Apple announced today that visionOS 2.4 will add some big features to the Vision Pro.
For starters, Apple Intelligence is coming to the Vision Pro. When I was at WWDC, I was surprised when there was no mention that Apple Intelligence would be added to visionOS. Since then, visionOS 2.0 and three subsequent releases have debuted without any sign of Apple Intelligence. However, that will change with the release of visionOS 2.4 in April.
Apple says visionOS 2.4 will include:
Writing Tools;
ChatGPT integration;
Genmoji;
Image Playground; and
Other unspecified features
Spatial Gallery. Source: Apple
In addition, visionOS will debut Spatial Gallery, a new Vision Pro app that Apple says will feature “a curated collection of spatial photos, spatial videos, and panoramas from artists, filmmakers, photographers, and more,” including new Immersive video titles.
Apple is also releasing a Vision Pro app for the iPhone that adds:
a new way to download apps and games from the App Store; discover experiences from Apple TV, Spatial Gallery, and more; easily find helpful tips; and quickly access information for their Vision Pro. Enhancements to Guest User make it easier than ever for users to share apps and experiences with family, friends, and colleagues using a nearby iPhone or iPad.
The Apple Vision Pro app. Source: Apple.
Apple says you’ll be able to queue a download from your iPhone for your Apple Vision Pro. I gave up trying to share my Vision Pro with my family almost immediately because it was such a cumbersome process, so better sharing will be welcome. If it works as promised, this sounds like a much better approach:
With new enhancements to Guest User in visionOS 2.4, users can start a Guest User session with their nearby iPhone or iPad. When their device is unlocked, they can choose which apps are accessible to their guest and start View Mirroring with AirPlay, making it easy to guide a guest through their Vision Pro experience.
It’s great to see Apple continue to push the Vision Pro forward with better user experiences, new content, and additional ways to enjoy its unique features.
YouTuber Two F Zero T has a first look at the STRUTT ev¹, a personal mobility device that was shown off at CES and can be controlled with the Vision Pro. The video, which I first saw linked by Apple’s Mike Stern on Mastodon, demonstrates the impressive tech packed into the STRUTT ev¹, including a unique integration with the Vision Pro. Thanks to Apple’s headset, users can navigate their surroundings with the Vision Pro’s eye and head tracking.
One of the things that’s easy to forget is that the Vision Pro builds on Apple’s years of accessibility research and development, which pioneered many of the interactions central to how people use it. With the release of the Vision Pro, developers working on new hardware like the STRUTT ev¹ can build on Apple’s innovation to offer an even richer feature set in their products. It’s a virtuous circle that benefits everyone. Apple’s products work better for more people, and companies like Strutt can build on that technology to offer an enhanced experience to their customers, too.
With a press release following an otherwise packed keynote at CES (which John and Brendon, my NPC co-hosts, attended in person last night), NVIDIA announced that their streaming service GeForce NOW is going to natively support the Apple Vision Pro…well, sort of.
There aren’t that many details in NVIDIA’s announcement, but the gist of it is that Vision Pro users will be able to stream games by visiting the GeForce NOW website when a new version launches “later this month”.
Get immersed in a new dimension of big-screen gaming as GeForce NOW brings AAA titles to life on Apple Vision Pro spatial computers, Meta Quest 3 and 3S and Pico virtual- and mixed-reality headsets. Later this month, these supported devices will give members access to an extensive library of games to stream through GeForce NOW by opening the browser to play.geforcenow.com when the newest app update, version 2.0.70, starts rolling out later this month.
This is all NVIDIA said in their announcement, which isn’t much, but we can speculate on a few things based on the existing limitations of visionOS.
For starters, the current version of Safari on visionOS does not support adding PWAs to the visionOS Home Screen. Given that the existing version of GeForce NOW requires saving a web app to begin the setup process, this either means that a) NVIDIA knows a visionOS software update in January will add the ability to save web apps or b) GeForce NOW won’t require that additional step to start playing on visionOS. The latter option seems more likely.
Second, as we covered last year, there is a workaround to play with GeForce NOW on visionOS, and that is the Nexus⁺ app. I’ve been using the Nexus⁺ app on my Vision Pro to stream Indiana Jones and other games from the cloud, and while the resolution is good enough1, what bothers me is the lack of HDR and Spatial Audio support (which should work with the Web Audio API in Safari for visionOS 2.0) in GeForce NOW when accessed from Nexus⁺’s built-in web browser.
The Nexus⁺ app supports ultra-wide aspect ratios, but HDR is nowhere to be found.
With all this in mind, I’m going to guess that, at a minimum, NVIDIA will support a PWA-free installation method in Safari for visionOS. I’m less optimistic about HDR and Spatial Audio, but as I gravitate more and more toward cloud streaming rather than local PC streaming2, I’d be happily proven wrong here.
My only question is: with the App Store’s “new” rules, why isn’t NVIDIA making a native GeForce NOW app for Apple platforms?
I’d love to know from people who know more about this stuff than I do whether Safari 18’s support for the WebRTC HEVC RFC 7789 RTP Payload Format makes a difference for GeForce NOW streaming or not. ↩︎
I’m actually thinking about selling my 4090 FE GPU in an effort to go all-in on cloud streaming and SteamOS in lieu of Windows in 2025. But this is a future topic for NPC. ↩︎
The Vision Pro has quickly become an essential item that I take onto every flight.
It’s a fantastic device to travel with—Be it by train or by plane, it offers an unparalleled opportunity to selectively tune out your environment and sink into an engaging activity like watching a movie or just working on your laptop.
In this blog post, I’ll outline what I’ve learned about the Vision Pro while traveling, explain some of the functionality, shine light onto its drawbacks, as well as assess how it fares against solutions like a phone or a laptop.
I haven’t been on a plane since I got my Vision Pro earlier this year; however, the next time I’ll be on a transatlantic flight, I plan on bringing mine and seeing how self-conscious I feel about it. Azad’s blog post has some great practical tips regarding using the Vision Pro during a flight, such as my favorite bit:
The problem is that for meals that require eyesight to coordinate (aka using a fork to pick up food from a plate), as soon as you look down at your food, the tracking often gets lost. This causes the movie to stop playing and for you to have to look forward for the tracking to re-initialize.
Additionally, the Vision Pro’s field of view is more horizontal than vertical (unlike most other VR headsets) which can make eating challenging, requiring me to fully tilt my head down to look at my food.
Alongside today’s debut of Submerged, the first scripted film captured in Apple Immersive Video, Apple also released a behind-the-scenes look at the film as well as details of immersive content coming to Apple Vision Pro users in the U.S. later this year. The Making of Submerged offers an inside look at the process behind shooting the immersive film. Writer/director Edward Berger and director of photography James Friend break down the unique considerations that went into telling a story in this new format.
Shot on location in Prague, Brussels, and Malta over three weeks, Submerged was filmed using a full-scale 23-ton submarine set made with real steel, brass, and metal that was modeled after WWII-era vessels. Significant portions of the set were built to withstand being fully submerged, and featured practical camera traps and special effects that were uniquely rigged to expose Apple Immersive Video cameras to sparks, steam, water, and fire without breaking viewers’ sense of immersion. Cast members who might appear out of frame or focus in a 2D feature were meticulously scripted, and participated in extensive stunt rehearsals, including freedive training in dive tanks and open water, to maintain continuity and realism.
It’s both fascinating and inspiring to see the ways the creative team behind Submerged adapted traditional filmmaking techniques to meet the challenges and embrace the opportunities of immersive video. The film itself is now available for Vision Pro owners to stream for free in the TV app in Australia, Canada, France, Germany, Hong Kong, Japan, Singapore, the U.K., and the U.S.
Apple has more immersive content on the way for Vision Pro users in the U.S. later this year, too, with global availability to follow. Here’s what fans of immersive video can expect:
2024 NBA All-Star Weekend: an immersive short film featuring footage from the event, including the Slam Dunk contest, the first-ever NBA vs. WNBA 3-Point Challenge, Stephen vs. Sabrina, the Rising Stars game, and highlights from the All-Star Game (available next Friday, October 17)
The Weeknd: an immersive music video for a new song from The Weeknd’s upcoming album, Hurry Up Tomorrow (available in November)
Adventure: a new episode of the extreme sports series centered on freediver Ant Williams’ attempt to best his record for the longest distance under ice with just a single breath (available in December)
Concert for One: a new concert series featuring intimate performances from the world’s biggest artists, starting with British singer-songwriter RAYE (available later this year)
The first episode of aerial travel series Elevated, featuring a trip around the islands of Hawai’i, is being made available outside the U.S. today. And early next year, Apple will release another episode of Elevated taking viewers on a crisp autumn journey through Maine’s winding coastlines and beautiful rivers, as well as a new episode of Adventure that follows sport climber Kai Lightner as he free-solo climbs over the rocky coves of Majorca, Spain.
All of these immersive video releases sound exciting, and I look forward to experiencing them. With lack of content being a primary concern among Vision Pro owners, it’s good to see new releases scheduled at a regular pace to keep users engaged. Based on today’s announcements, we have lots of immersive content to look forward to in the coming months.
Today, Apple released the trailer for Submerged, a short film from director Edward Berger (All Quiet on the Western Front) that the company is touting as the first scripted film created using its Immersive Video format. Available free this Thursday exclusively on Apple Vision Pro, the 17-minute thriller follows a WWII submarine crew combating a harrowing torpedo attack.
With its 3D, 8K footage displayed in a 180-degree field of view combined with Spatial Audio, Apple Immersive Video is one of the highlights of the Vision Pro experience. Thus far, we’ve only seen it used in nonfictional contexts, including nature, travel, and sports. It will be intriguing to see how the format works for a fictional, scripted story.
I’m glad the company is continuing to release immersive content for Vision Pro owners to enjoy at what seems to be a monthly pace, and I hope to see more storytellers given the opportunity to show what’s possible with this new format soon. It’s early days, and the potential is certainly there for finding creative new ways to engage audiences. I’m excited to see what happens when Immersive Video takes the plunge into fiction.
In the lead-up to this year’s WWDC, it was hard to predict what an update to visionOS would look like. After all, the initial version had only shipped four months earlier when Apple Vision Pro became available for purchase in the United States. Given how late in the software cycle visionOS 1 shipped, it was reasonable to wonder if there would be a visionOS 2 announced at all, and if so, how much it could realistically add to an operating system that had just debuted the previous quarter.
Of course, Apple’s software cycle waits for no one, so like watchOS before it, visionOS is receiving a 2.0 version rapidly on the heels of its initial release. But the shortened development window doesn’t mean that this update isn’t a significant one. I believe that the 2.0 moniker is well deserved based on the features and enhancements included in this release, especially given the quieter updates across all of Apple’s platforms this year in the wake of Apple Intelligence.
visionOS 2 moves spatial computing forward with an array of welcome quality-of-life improvements, deeper integration with some of Apple’s other platforms, additional tools for developers to create spatial experiences, system app updates in line with Apple’s other platforms, and a new way to experience photos that you have to see to believe. The combination of user experience refinements and new features makes for a solid update that Vision Pro users are definitely going to notice and enjoy.
Some of the changes we’ll dive into feel so obvious that you might wonder why they weren’t included in visionOS to begin with. Having used Vision Pro almost daily since it was released, I fully understand the sentiment. But then I remember that the iPhone didn’t gain the ability to copy and paste text until iPhone OS 3, and I’m reminded that developing new platforms takes time – even for a company as big as Apple.
So while some might seem basic, many of the changes included in visionOS 2 improve users’ experiences in significant ways every time they interact with the platform. The end result is a smoother, more intuitive operating system that will delight Vision Pro believers and, if Apple has its way, convince more skeptics to take the plunge into spatial computing.
