Posts tagged with "Vision Pro"

Vision Pro to Gain Apple Intelligence, Spatial Gallery App, and Companion iPhone App

Source: Apple.

Source: Apple.

Apple announced today that visionOS 2.4 will add some big features to the Vision Pro.

For starters, Apple Intelligence is coming to the Vision Pro. When I was at WWDC, I was surprised when there was no mention that Apple Intelligence would be added to visionOS. Since then, visionOS 2.0 and three subsequent releases have debuted without any sign of Apple Intelligence. However, that will change with the release of visionOS 2.4 in April.

Apple says visionOS 2.4 will include:

  • Writing Tools;
  • ChatGPT integration;
  • Genmoji;
  • Image Playground; and
  • Other unspecified features
Spatial Gallery. Source: Apple

Spatial Gallery. Source: Apple

In addition, visionOS will debut Spatial Gallery, a new Vision Pro app that Apple says will feature “a curated collection of spatial photos, spatial videos, and panoramas from artists, filmmakers, photographers, and more,” including new Immersive video titles.

Apple is also releasing a Vision Pro app for the iPhone that adds:

a new way to download apps and games from the App Store; discover experiences from Apple TV, Spatial Gallery, and more; easily find helpful tips; and quickly access information for their Vision Pro. Enhancements to Guest User make it easier than ever for users to share apps and experiences with family, friends, and colleagues using a nearby iPhone or iPad.

The Apple Vision Pro app. Source: Apple.

The Apple Vision Pro app. Source: Apple.

Apple says you’ll be able to queue a download from your iPhone for your Apple Vision Pro. I gave up trying to share my Vision Pro with my family almost immediately because it was such a cumbersome process, so better sharing will be welcome. If it works as promised, this sounds like a much better approach:

With new enhancements to Guest User in visionOS 2.4, users can start a Guest User session with their nearby iPhone or iPad. When their device is unlocked, they can choose which apps are accessible to their guest and start View Mirroring with AirPlay, making it easy to guide a guest through their Vision Pro experience.

It’s great to see Apple continue to push the Vision Pro forward with better user experiences, new content, and additional ways to enjoy its unique features.

The Vision Pro Lends a Hand to the STRUTT ev¹ Personal Mobility Device→

YouTuber Two F Zero T has a first look at the STRUTT ev¹, a personal mobility device that was shown off at CES and can be controlled with the Vision Pro. The video, which I first saw linked by Apple’s Mike Stern on Mastodon, demonstrates the impressive tech packed into the STRUTT ev¹, including a unique integration with the Vision Pro. Thanks to Apple’s headset, users can navigate their surroundings with the Vision Pro’s eye and head tracking.

One of the things that’s easy to forget is that the Vision Pro builds on Apple’s years of accessibility research and development, which pioneered many of the interactions central to how people use it. With the release of the Vision Pro, developers working on new hardware like the STRUTT ev¹ can build on Apple’s innovation to offer an even richer feature set in their products. It’s a virtuous circle that benefits everyone. Apple’s products work better for more people, and companies like Strutt can build on that technology to offer an enhanced experience to their customers, too.

Permalink

NVIDIA Announces GeForce NOW Support Coming to Safari on Vision Pro Later This Month

With a press release following an otherwise packed keynote at CES (which John and Brendon, my NPC co-hosts, attended in person last night), NVIDIA announced that their streaming service GeForce NOW is going to natively support the Apple Vision Pro…well, sort of.

There aren’t that many details in NVIDIA’s announcement, but the gist of it is that Vision Pro users will be able to stream games by visiting the GeForce NOW website when a new version launches “later this month”.

Get immersed in a new dimension of big-screen gaming as GeForce NOW brings AAA titles to life on Apple Vision Pro spatial computers, Meta Quest 3 and 3S and Pico virtual- and mixed-reality headsets. Later this month, these supported devices will give members access to an extensive library of games to stream through GeForce NOW by opening the browser to play.geforcenow.com when the newest app update, version 2.0.70, starts rolling out later this month.

This is all NVIDIA said in their announcement, which isn’t much, but we can speculate on a few things based on the existing limitations of visionOS.

For starters, the current version of Safari on visionOS does not support adding PWAs to the visionOS Home Screen. Given that the existing version of GeForce NOW requires saving a web app to begin the setup process, this either means that a) NVIDIA knows a visionOS software update in January will add the ability to save web apps or b) GeForce NOW won’t require that additional step to start playing on visionOS. The latter option seems more likely.

Second, as we covered last year, there is a workaround to play with GeForce NOW on visionOS, and that is the Nexus⁺ app. I’ve been using the Nexus⁺ app on my Vision Pro to stream Indiana Jones and other games from the cloud, and while the resolution is good enough1, what bothers me is the lack of HDR and Spatial Audio support (which should work with the Web Audio API in Safari for visionOS 2.0) in GeForce NOW when accessed from Nexus⁺’s built-in web browser.

The Nexus⁺ app supports ultra-wide aspect ratios, but HDR is nowhere to be found.

The Nexus⁺ app supports ultra-wide aspect ratios, but HDR is nowhere to be found.

With all this in mind, I’m going to guess that, at a minimum, NVIDIA will support a PWA-free installation method in Safari for visionOS. I’m less optimistic about HDR and Spatial Audio, but as I gravitate more and more toward cloud streaming rather than local PC streaming2, I’d be happily proven wrong here.

My only question is: with the App Store’s “new” rules, why isn’t NVIDIA making a native GeForce NOW app for Apple platforms?

  1. I’d love to know from people who know more about this stuff than I do whether Safari 18’s support for the WebRTC HEVC RFC 7789 RTP Payload Format makes a difference for GeForce NOW streaming or not. ↩︎
  2. I’m actually thinking about selling my 4090 FE GPU in an effort to go all-in on cloud streaming and SteamOS in lieu of Windows in 2025. But this is a future topic for NPC. ↩︎

Traveling with the Vision Pro→

Excellent deep dive into the Vision Pro’s travel capabilities by Azad Balabanian, who’s used Apple’s headset extensively on various flights:

The Vision Pro has quickly become an essential item that I take onto every flight.

It’s a fantastic device to travel with—Be it by train or by plane, it offers an unparalleled opportunity to selectively tune out your environment and sink into an engaging activity like watching a movie or just working on your laptop.

In this blog post, I’ll outline what I’ve learned about the Vision Pro while traveling, explain some of the functionality, shine light onto its drawbacks, as well as assess how it fares against solutions like a phone or a laptop.

I haven’t been on a plane since I got my Vision Pro earlier this year; however, the next time I’ll be on a transatlantic flight, I plan on bringing mine and seeing how self-conscious I feel about it. Azad’s blog post has some great practical tips regarding using the Vision Pro during a flight, such as my favorite bit:

The problem is that for meals that require eyesight to coordinate (aka using a fork to pick up food from a plate), as soon as you look down at your food, the tracking often gets lost. This causes the movie to stop playing and for you to have to look forward for the tracking to re-initialize.

Additionally, the Vision Pro’s field of view is more horizontal than vertical (unlike most other VR headsets) which can make eating challenging, requiring me to fully tilt my head down to look at my food.

If you plan on bringing the Vision Pro on a flight with you (personally, I recommend checking out Waterfield’s compact carrying case), don’t miss Azad’s experience and advice.

Permalink

Apple Releases Immersive Short Film Submerged and Announces Upcoming Immersive Content

Alongside today’s debut of Submerged, the first scripted film captured in Apple Immersive Video, Apple also released a behind-the-scenes look at the film as well as details of immersive content coming to Apple Vision Pro users in the U.S. later this year. The Making of Submerged offers an inside look at the process behind shooting the immersive film. Writer/director Edward Berger and director of photography James Friend break down the unique considerations that went into telling a story in this new format.

Apple offered even more insight into the creative process behind Submerged:

Shot on location in Prague, Brussels, and Malta over three weeks, Submerged was filmed using a full-scale 23-ton submarine set made with real steel, brass, and metal that was modeled after WWII-era vessels. Significant portions of the set were built to withstand being fully submerged, and featured practical camera traps and special effects that were uniquely rigged to expose Apple Immersive Video cameras to sparks, steam, water, and fire without breaking viewers’ sense of immersion. Cast members who might appear out of frame or focus in a 2D feature were meticulously scripted, and participated in extensive stunt rehearsals, including freedive training in dive tanks and open water, to maintain continuity and realism.

It’s both fascinating and inspiring to see the ways the creative team behind Submerged adapted traditional filmmaking techniques to meet the challenges and embrace the opportunities of immersive video. The film itself is now available for Vision Pro owners to stream for free in the TV app in Australia, Canada, France, Germany, Hong Kong, Japan, Singapore, the U.K., and the U.S.

Apple has more immersive content on the way for Vision Pro users in the U.S. later this year, too, with global availability to follow. Here’s what fans of immersive video can expect:

  • 2024 NBA All-Star Weekend: an immersive short film featuring footage from the event, including the Slam Dunk contest, the first-ever NBA vs. WNBA 3-Point Challenge, Stephen vs. Sabrina, the Rising Stars game, and highlights from the All-Star Game (available next Friday, October 17)
  • The Weeknd: an immersive music video for a new song from The Weeknd’s upcoming album, Hurry Up Tomorrow (available in November)
  • Adventure: a new episode of the extreme sports series centered on freediver Ant Williams’ attempt to best his record for the longest distance under ice with just a single breath (available in December)
  • Concert for One: a new concert series featuring intimate performances from the world’s biggest artists, starting with British singer-songwriter RAYE (available later this year)

The first episode of aerial travel series Elevated, featuring a trip around the islands of Hawai’i, is being made available outside the U.S. today. And early next year, Apple will release another episode of Elevated taking viewers on a crisp autumn journey through Maine’s winding coastlines and beautiful rivers, as well as a new episode of Adventure that follows sport climber Kai Lightner as he free-solo climbs over the rocky coves of Majorca, Spain.

All of these immersive video releases sound exciting, and I look forward to experiencing them. With lack of content being a primary concern among Vision Pro owners, it’s good to see new releases scheduled at a regular pace to keep users engaged. Based on today’s announcements, we have lots of immersive content to look forward to in the coming months.

Submerged, Apple’s First Scripted Immersive Film, Is Coming This Thursday→

Today, Apple released the trailer for Submerged, a short film from director Edward Berger (All Quiet on the Western Front) that the company is touting as the first scripted film created using its Immersive Video format. Available free this Thursday exclusively on Apple Vision Pro, the 17-minute thriller follows a WWII submarine crew combating a harrowing torpedo attack.

With its 3D, 8K footage displayed in a 180-degree field of view combined with Spatial Audio, Apple Immersive Video is one of the highlights of the Vision Pro experience. Thus far, we’ve only seen it used in nonfictional contexts, including nature, travel, and sports. It will be intriguing to see how the format works for a fictional, scripted story.

I’m glad the company is continuing to release immersive content for Vision Pro owners to enjoy at what seems to be a monthly pace, and I hope to see more storytellers given the opportunity to show what’s possible with this new format soon. It’s early days, and the potential is certainly there for finding creative new ways to engage audiences. I’m excited to see what happens when Immersive Video takes the plunge into fiction.

Permalink

visionOS 2: The MacStories Review

In the lead-up to this year’s WWDC, it was hard to predict what an update to visionOS would look like. After all, the initial version had only shipped four months earlier when Apple Vision Pro became available for purchase in the United States. Given how late in the software cycle visionOS 1 shipped, it was reasonable to wonder if there would be a visionOS 2 announced at all, and if so, how much it could realistically add to an operating system that had just debuted the previous quarter.

Of course, Apple’s software cycle waits for no one, so like watchOS before it, visionOS is receiving a 2.0 version rapidly on the heels of its initial release. But the shortened development window doesn’t mean that this update isn’t a significant one. I believe that the 2.0 moniker is well deserved based on the features and enhancements included in this release, especially given the quieter updates across all of Apple’s platforms this year in the wake of Apple Intelligence.

visionOS 2 moves spatial computing forward with an array of welcome quality-of-life improvements, deeper integration with some of Apple’s other platforms, additional tools for developers to create spatial experiences, system app updates in line with Apple’s other platforms, and a new way to experience photos that you have to see to believe. The combination of user experience refinements and new features makes for a solid update that Vision Pro users are definitely going to notice and enjoy.

Some of the changes we’ll dive into feel so obvious that you might wonder why they weren’t included in visionOS to begin with. Having used Vision Pro almost daily since it was released, I fully understand the sentiment. But then I remember that the iPhone didn’t gain the ability to copy and paste text until iPhone OS 3, and I’m reminded that developing new platforms takes time – even for a company as big as Apple.

So while some might seem basic, many of the changes included in visionOS 2 improve users’ experiences in significant ways every time they interact with the platform. The end result is a smoother, more intuitive operating system that will delight Vision Pro believers and, if Apple has its way, convince more skeptics to take the plunge into spatial computing.

Let’s jump into what visionOS 2 has to offer.

Read more

    Apple Announces New Immersive Video Content for the Vision Pro

    Source: Apple.

    Source: Apple.

    Beginning today, Apple says it will begin releasing a series of films, documentaries, music, sports, travel, and other video content for the Vision Pro.

    First up is ‘Hot Air Balloons,’ which will be available beginning at 6 PM Pacific today, and is part of a series dubbed Boundless. That will be followed in August by a new installment of Wild Life, and a new travel series called Elevated in September. Later this year, Apple will release an immersive concert experience with The Weeknd, a scripted short film called Submerged by director Edward Berger, a behind-the-scenes and court-side look at the 2024 NBA All-Star Weekend, and Big-Wave Surfing, part of a new sports series with Red Bull.

    According to Tor Myhren, Apple’s vice president of Marketing Communications:

    Apple Immersive Video is a groundbreaking leap forward for storytelling, offering Apple Vision Pro users remarkable experiences with an unparalleled sense of realism and immersion. From soaring over volcanoes in Hawaii and surfing huge waves in Tahiti, to enjoying performances by the world’s biggest artists and athletes from all-new perspectives, Apple Immersive Video revolutionizes the way people experience places, stories, sports, and more by making viewers feel like they’re truly there. It’s the next generation of visual storytelling, and we’re excited to bring it to more people around the world.

    I’m really looking forward to these new immersive videos. There’s been a real dearth of content produced for the Vision Pro since its release, but with the device available in more countries, it’s great to see new content available for the growing audience.

    visionOS 2: The MacStories Overview

    Today, Apple revealed visionOS 2, the first major upgrade to the software powering Apple Vision Pro. The update, coming this fall, will include quality-of-life improvements, new ways to create and share spatial content, additional tools for developers, and more.

    Interacting with the Home View and Control Center will be easier than before thanks to new system-wide gestures. Holding your hand up and tapping your fingers together will bring up the Home View, while flipping your hand over will bring up a time and battery status popover that also acts as a Control Center launcher. These gestures are simpler than the previous methods of accessing these views, which required pushing the Digital Crown for the Home View and looking upwards to reveal the Control Center launcher. Apps in the Home View can now be rearranged, and compatible iPad and iPhone apps are no longer constrained to a single folder.

    The new gestures for accessing the Home View and Control Center in visionOS 2.Replay

    Mac Virtual Display is expanding later this year to support a higher resolution and larger size, resulting in a display roughly equivalent to two 4K monitors side by side. The experience of using peripherals with Apple Vision Pro will be improved with Magic Keyboard passthrough in immersive environments and support for mice. Guest User information will be saved for 30 days so users’ friends and family can use Apple Vision Pro without going through the setup process repeatedly, and Travel Mode will be supported on trains in addition to airplanes.

    Read more