Posts tagged with "visionOS"

Apple Vision Glasses Will Be Irresistible→

I found myself nodding in agreement from beginning to end with this story by Lachlan Campbell, who, after a year of Vision Pro, imagines what future Apple Vision glasses may be able to do and how they’d reshape our societal norms:

I’ve written about my long-term belief in spatial computing, and how visionOS 2 made small but notable progress. The pieces have clicked into place more recently for me for what an AR glasses version of Apple Vision would look like, and how it will change us. We don’t have the technology, hardware-wise, to build this product today, or we’d already be wearing it. We need significant leaps in batteries, mobile silicon, and displays to make this product work. Leaps in AI assistance, cameras, and computer vision would make this product better, too. But the industry is hard at work at all of these problems. This product is coming.

The basic pitch: augmented reality glasses with transparent lenses that can project more screen than you could ever own, wherever you are. The power of real software like iPad/Mac, an always-on intelligent assistant, POV photos/video/audio, and listening to audio without headphones. Control it like Apple Vision Pro with your eyes, hands, and voice, optionally pairing accessories (primarily AirPods and any of stylus/keyboard/trackpad/mice work for faster/more precise inputs). It’s cellular (with an Apple-designed modem) and entirely wireless. It combines the ideas of ambient computing that Humane (RIP) and Meta Ray-Bans have begun, including a wearable assistant, POV photography, and ambient audio with everything you love about your current Apple products.

I may be stating the obvious here, but I fundamentally believe that headsets are a dead end and glasses are the ultimate form factor we should be striving for. Or let me put it another way: every time I use visionOS, I remember how futuristic everything about it still feels…and how much I wish I was looking at it through glasses instead.

There’s a real possibility we may have Apple glasses (and an Apple foldable?) by 2030, and I wish I could just skip ahead five years now. As Lachlan argues, we’re marching toward all of this.

Permalink

NVIDIA Announces GeForce NOW Support Coming to Safari on Vision Pro Later This Month

With a press release following an otherwise packed keynote at CES (which John and Brendon, my NPC co-hosts, attended in person last night), NVIDIA announced that their streaming service GeForce NOW is going to natively support the Apple Vision Pro…well, sort of.

There aren’t that many details in NVIDIA’s announcement, but the gist of it is that Vision Pro users will be able to stream games by visiting the GeForce NOW website when a new version launches “later this month”.

Get immersed in a new dimension of big-screen gaming as GeForce NOW brings AAA titles to life on Apple Vision Pro spatial computers, Meta Quest 3 and 3S and Pico virtual- and mixed-reality headsets. Later this month, these supported devices will give members access to an extensive library of games to stream through GeForce NOW by opening the browser to play.geforcenow.com when the newest app update, version 2.0.70, starts rolling out later this month.

This is all NVIDIA said in their announcement, which isn’t much, but we can speculate on a few things based on the existing limitations of visionOS.

For starters, the current version of Safari on visionOS does not support adding PWAs to the visionOS Home Screen. Given that the existing version of GeForce NOW requires saving a web app to begin the setup process, this either means that a) NVIDIA knows a visionOS software update in January will add the ability to save web apps or b) GeForce NOW won’t require that additional step to start playing on visionOS. The latter option seems more likely.

Second, as we covered last year, there is a workaround to play with GeForce NOW on visionOS, and that is the Nexus⁺ app. I’ve been using the Nexus⁺ app on my Vision Pro to stream Indiana Jones and other games from the cloud, and while the resolution is good enough1, what bothers me is the lack of HDR and Spatial Audio support (which should work with the Web Audio API in Safari for visionOS 2.0) in GeForce NOW when accessed from Nexus⁺’s built-in web browser.

The Nexus⁺ app supports ultra-wide aspect ratios, but HDR is nowhere to be found.

The Nexus⁺ app supports ultra-wide aspect ratios, but HDR is nowhere to be found.

With all this in mind, I’m going to guess that, at a minimum, NVIDIA will support a PWA-free installation method in Safari for visionOS. I’m less optimistic about HDR and Spatial Audio, but as I gravitate more and more toward cloud streaming rather than local PC streaming2, I’d be happily proven wrong here.

My only question is: with the App Store’s “new” rules, why isn’t NVIDIA making a native GeForce NOW app for Apple platforms?

  1. I’d love to know from people who know more about this stuff than I do whether Safari 18’s support for the WebRTC HEVC RFC 7789 RTP Payload Format makes a difference for GeForce NOW streaming or not. ↩︎
  2. I’m actually thinking about selling my 4090 FE GPU in an effort to go all-in on cloud streaming and SteamOS in lieu of Windows in 2025. But this is a future topic for NPC. ↩︎

Shareshot 1.0: Pixel Perfect Screenshots, Beautifully Presented

I’ve used Federico’s Apple Frames shortcut to add device frames to screenshots for years. It’s a great way to quickly process multiple screenshots and upload them to the MacStories CDN with minimal effort. But as great as Shortcuts is for simplifying this sort of task, there are advantages to using a native app instead.

Editing screenshots on the iPad.

Editing screenshots on the iPad.

Shareshot is an app for iPhone, iPad, and Vision Pro that occupies a lot of the same feature space as Apple Frames, using Apple hardware to frame screenshots. However, the app simultaneously does more and a little less than Federico’s shortcut. That one missing feature means that I won’t be abandoning Apple Frames, but because of the things the app can do that Apple Frames can’t, Shareshot will be joining the shortcut as a utility I expect to use a lot.

Let’s dig into what makes Shareshot shine.

Read more

GoodLinks 2.0: The Automation-Focused Read-Later App I’ve Always Wanted

One of my greatest frustrations with read-later apps is how hard most make it to get your data out on your terms. Few allow you to export links using Shortcuts or some other system, and even fewer offer to do the same with highlights – until now. With version 2.0, GoodLinks adds highlighting and note-taking combined with excellent Shortcuts support, giving users full access and flexibility to incorporate saved URLs, highlights, and notes into their workflows however they want.

Thanks to Obsidian’s deep catalog of plugins from third-party developers, it’s been possible to import highlights from read-later apps like Readwise Reader and Amazon’s Kindle app for some time. Those are good solutions when I’m working in Obsidian, but both I and our readers use lots of different apps. That’s why I was so glad to see GoodLinks (available for the iPhone, iPad, and Mac) get this major 2.0 revision that transforms it from a place to save links and articles to a more well-rounded research tool, thanks to highlighting and notes.

Read more

visionOS 2: The MacStories Overview

Today, Apple revealed visionOS 2, the first major upgrade to the software powering Apple Vision Pro. The update, coming this fall, will include quality-of-life improvements, new ways to create and share spatial content, additional tools for developers, and more.

Interacting with the Home View and Control Center will be easier than before thanks to new system-wide gestures. Holding your hand up and tapping your fingers together will bring up the Home View, while flipping your hand over will bring up a time and battery status popover that also acts as a Control Center launcher. These gestures are simpler than the previous methods of accessing these views, which required pushing the Digital Crown for the Home View and looking upwards to reveal the Control Center launcher. Apps in the Home View can now be rearranged, and compatible iPad and iPhone apps are no longer constrained to a single folder.

The new gestures for accessing the Home View and Control Center in visionOS 2.Replay

Mac Virtual Display is expanding later this year to support a higher resolution and larger size, resulting in a display roughly equivalent to two 4K monitors side by side. The experience of using peripherals with Apple Vision Pro will be improved with Magic Keyboard passthrough in immersive environments and support for mice. Guest User information will be saved for 30 days so users’ friends and family can use Apple Vision Pro without going through the setup process repeatedly, and Travel Mode will be supported on trains in addition to airplanes.

Read more

Sequel 2.3’s Magic Lookup Is a Perfect Example of How Apps Should Integrate AI

If you haven’t tried Sequel to track your media queues, you’re missing out and have some catching up to do. There’s a reason we gave Sequel 2.0 the MacStories Selects Best App Update award last year. The app, which is available on the iPhone, iPad, Mac, and Vision Pro, is one of the very best media trackers in a very crowded field.

One way to get up to speed with Sequel’s core features is my review of version 2.0, which I won’t repeat here. Since that review, developer Romain Lefebvre has released updates that continue to improve the app in meaningful ways. However, today’s version 2.3 release is special because it’s one of the first apps I’ve tried that uses AI in a truly useful, non-gimmicky way. Plus, the update adds Shortcuts integration, allowing users to automate the app in new ways, as well as a Markdown-compatible note-taking feature and new layout and search options.

Read more

What If…? – An Immersive Story Will Debut May 30th

Source: Disney+.

Source: Disney+.

Earlier this month, I linked to What If…? – An Immersive Story, which Marvel Studios and ILM Immersive developed for Disney+. The project is an Apple Vision Pro exclusive for which there were few details originally. Now, however, we have a trailer and a launch date. What If…? – An Immersive Story will be available to Disney+ subscribers in just over one week on May 30th:

The trailer’s description explains the project’s premise:

The Multiverse is in danger and The Watcher needs your help. Dangerous variants are hunting Infinity Stones and altering the fate of not only their realities, but yours as well. To save the fate of the Multiverse, you’ll need to use your own hands to learn mystic spells, defend your allies in epic battles, and more. But, be careful… everything might not be all that it seems.

Source: Disney+.

Source: Disney+.

I can’t wait to try What If…? – An Immersive Story. There hasn’t been a lot of new immersive content since the Vision Pro’s launch. However, along with the appearance of Parkour in the TV app, which is set for release in a couple of days, the pace seems to be picking up.

Apple Marks Global Accessibility Awareness Day with a Preview of OS Features Coming Later This Year

Source: Apple.

Source: Apple.

Thursday is Global Accessibility Awareness Day, and to mark the occasion, Apple has previewed several new accessibility features coming to its OSes later this year. Although this accessibility preview has become an annual affair, this year’s preview is more packed than most years, with a wide variety of features for navigating UIs, automating tasks, interacting with Siri and CarPlay, enabling live captions in visionOS, and more. Apple hasn’t announced when these features will debut, but if past years are any indication, most should be released in the fall as part of the annual OS release cycle.

Eye Tracking

Often, Apple’s work in one area lends itself to new accessibility features in another. With Eye Tracking in iOS and iPadOS, the connection to the company’s work on visionOS is clear. The feature will allow users to look at UI elements on the iPhone and iPad, and the front-facing camera – combined with a machine learning model – will follow their gaze, moving the selection as what they look at changes. No additional hardware is necessary.

Eye Tracking also works with Dwell, meaning that when a user pauses their gaze on an interface element, it will be clicked. The feature, which requires a one-time calibration setup process, will work with Apple’s apps, as well as third-party apps, on iPhones and iPads with an A12 Bionic chip or newer.

Vocal Shortcuts

Source: Apple.

Source: Apple.

Vocal Shortcuts provide a way to define custom utterances that launch shortcuts and other tasks. The phrases are defined on-device for maximum privacy using a process similar to Personal Voice. The feature is like triggering shortcuts with Siri, but it doesn’t require an assistant trigger word or phrase.

Read more

Vision Pro App Spotlight: Seasons Weaves Immersive Conditions Into a Comprehensive Weather App

Seasons is the sort of weather app I’d hoped for ever since I ordered my Vision Pro. It’s a unique mix of detailed forecast data combined with an immersive spatial computing experience. There’s a gee-whiz, proof-of-concept aspect to the app, but at its core, Seasons is a serious weather app and a spatial widget that’s a pleasure to incorporate into an everyday Vision Pro workflow.

Read more