The M3 Ultra Mac Studio for Local LLMs→

Speaking of the new Mac Studio and Apple making the best computers for AI: this is a terrific overview by Max Weinbach about the new M3 Ultra chip and its real-world performance with various on-device LLMs:

The Mac I’ve been using for the past few days is the Mac Studio with M3 Ultra SoC, 32-core CPU, 80-core GPU, 256GB Unified Memory (192GB usable for VRAM), and 4TB SSD. It’s the fastest computer I have. It is faster in my workflows for even AI than my gaming PC (which will be used for comparisons below; it has an Intel i9 13900K, RTX 5090, 64GB of DDR5, and a 2TB NVMe SSD).

It’s a very technical read, but the comparison between the M3 Ultra and a vanilla (non-optimized) RTX 5090 is mind-blogging to me. According to Weinbach, it all comes down to Apple’s MLX framework:

I’ll keep it brief; the LLM performance is essentially as good as you’ll get for the majority of models. You’ll be able to run better models faster with larger context windows on a Mac Studio or any Mac with Unified Memory than essentially any PC on the market. This is simply the inherent benefit of not only Apple Silicon but Apple’s MLX framework (the reason we can efficiently run the models without preloading KV Cache into memory, as well as generate tokens faster as context windows grow).

In case you’re not familiar, MLX is Apple’s open-source framework that – I’m simplifying – optimizes training and serving models on Apple Silicon’s unified memory architecture. It is a wonderful project with over 1,600 community models available for download.

As Weinbach concludes:

I see one of the best combos any developer can do as: M3 Ultra Mac Studio with an Nvidia 8xH100 rented rack. Hopper and Blackwell are outstanding for servers, M3 Ultra is outstanding for your desk. Different machines for a different use, while it’s fun to compare these for sport, that’s not the reality.⁠⁠

There really is no competition for an AI workstation today. The reality is, the only option is a Mac Studio.

Don’t miss the benchmarks in the story.

Permalink

Is Apple Shipping the Best AI Computers?→

For all the criticism (mine included) surrounding Apple’s delay of various Apple Intelligence features, I found this different perspective by Ben Thompson fascinating and worth considering:

What that means in practical terms is that Apple just shipped the best consumer-grade AI computer ever. A Mac Studio with an M3 Ultra chip and 512GB RAM can run a 4-bit quantized version of DeepSeek R1 — a state-of-the-art open-source reasoning model — right on your desktop. It’s not perfect — quantization reduces precision, and the memory bandwidth is a bottleneck that limits performance — but this is something you simply can’t do with a standalone Nvidia chip, pro or consumer. The former can, of course, be interconnected, giving you superior performance, but that costs hundreds of thousands of dollars all-in; the only real alternative for home use would be a server CPU and gobs of RAM, but that’s even slower, and you have to put it together yourself. Apple didn’t, of course, explicitly design the M3 Ultra for R1; the architectural decisions undergirding this chip were surely made years ago. In fact, if you want to include the critical decision to pursue a unified memory architecture, then your timeline has to extend back to the late 2000s, whenever the key architectural decisions were made for Apple’s first A4 chip, which debuted in the original iPad in 2010. Regardless, the fact of the matter is that you can make a strong case that Apple is the best consumer hardware company in AI, and this week affirmed that reality.

Anecdotally speaking, based on the people who cover AI that I follow these days, it seems there are largely two buckets of folks who are into local, on-device models: those who have set up pricey NVIDIA rigs at home for their CUDA cores (the vast minority); and – the undeniable majority – those who run a spectrum of local models on their Macs of different shapes and configurations (usually, MacBook Pros). If you have to run high-end, performance-intensive local models for academic or scientific workflows on a desktop, the M3 Ultra Mac Studio sounds like an absolute winner.

However, I’d point out that – again, as far as local, on-device models are concerned – Apple is not shipping the best possible hardware on smartphones.

While the entire iPhone 16 lineup is stuck on 8 GB of RAM (and we know how memory-hungry these models can be), Android phones with at least 12 GB or 16 GB of RAM are becoming pretty much the norm now, especially in flagship territory. Even better in Android land, what are being advertised as “gaming phones” with a whopping 24 GB of RAM (such as the ASUS ROG Phone 9 Pro or the RedMagic 10 Pro) may actually make for compelling pocket computers to run smaller, distilled versions of DeepSeek, LLama, or Mistral with better performance than current iPhones.

Interestingly, I keep going back to this quote from Mark Gurman’s latest report on Apple’s AI challenges:

There are also concerns internally that fixing Siri will require having more powerful AI models run on Apple’s devices. That could strain the hardware, meaning Apple either has to reduce its set of features or make the models run more slowly on current or older devices. It would also require upping the hardware capabilities of future products to make the features run at full strength.

Given Apple’s struggles, their preference for a hybrid on-device/server-based AI system, and the market’s evolution on Android, I don’t think Apple can afford to ship 8 GB on iPhones for much longer if they’re serious about AI and positioning their hardware as the best consumer-grade AI computers.

Permalink

The ‘e’ Is for Elemental

Source: Apple.

Source: Apple.

For the past 10 days, I’ve been testing the iPhone 16e – but not in the way I typically test new hardware. You see, I didn’t buy the iPhone 16e to make calls, send email, surf the web, post to social media, or anything else, really. Instead, I got it for one thing: the camera.

Read more

DEVONthink: Store, Organize, and Work the Smart Way [Sponsor]

DEVONthink 3 is the all-in-one solution to organizing and retrieving documents across macOS, iOS, and iPadOS. By supporting numerous file formats, it’s the perfect solution to bring your work together in one powerful hub.

What sets DEVONthink apart is its many information retrieval features. The app learns from the way you use it, so it can automatically classify and tag your documents. Plus, with advanced Boolean operators and smart groups of saved searches, it’s simple to create refined searches that you can return to whenever you need them.

DEVONthink has lightning-fast search, too. It uses advanced AI to go beyond simple keyword searches, finding the most relevant documents instantly.

The app is also the perfect companion for web-based research. You can automatically pull from RSS feeds, save webpages in a variety of formats, including Markdown, and then highlight, comment, and take notes. The app can even suggest connections between documents and provide contextual recommendations using its integrated AI.

With OCR technology built in, DEVONthink is also perfect for going paperless. Just scan in your documents, and DEVONthink takes care of making them text-searchable.

Everything in your DEVONthink database can sync between multiple Macs, as well as iPhones and iPads. The app also uses industry-grade encryption to secure your documents and offers easy export of documents in multiple formats.

There’s never been a better time to bring order to your information. DEVONtechnologies is offering 10% off on all versions of DEVONthink to MacStories readers for a limited time. Visit DEVONtechnologies’ website today to learn more about how DEVONthink can help you gain the upper hand on your data and to take advantage of this great offer.

Our thanks to DEVONtechnologies for sponsoring MacStories this week.

Notes on the Apple Intelligence Delay

Simon Willison, one of the more authoritative independent voices in the LLM space right now, published a good theory on what may have happened with Apple’s delay of Apple Intelligence’s Siri personalization features:

I have a hunch that this delay might relate to security.

These new Apple Intelligence features involve Siri responding to requests to access information in applications and then perform actions on the user’s behalf.

This is the worst possible combination for prompt injection attacks! Any time an LLM-based system has access to private data, tools it can call and potentially malicious instructions (like emails and text messages from untrusted strangers) there’s a risk that an attacker might subvert those tools and use them to damage or exfiltration a user’s data.

Willison has been writing about prompt injection attacks since 2023. We know that Mail’s AI summaries were (at least initially?) sort of susceptible to prompt injections (using hidden HTML elements), as were Writing Tools during the beta period. It’s scary to imagine what would happen with a well-crafted prompt injection when the attack’s surface area becomes the entire assistant directly plugged into your favorite apps with your data. But then again, one has to wonder why these features were demoed at all at Apple’s biggest software event last year and if those previews – absent a real, in-person event – were actually animated prototypes.

On this note, I disagree with Jason Snell’s idea that previewing Apple Intelligence last year was a good move no matter what. Are we sure that “nobody is looking” at Apple’s position in the AI space right now and that Siri isn’t continuing down its path of damaging Apple’s software reputation, like MobileMe did? As a reminder, the iPhone 16 lineup was advertised as “built for Apple Intelligence” in commercials, interviews, and Apple’s website.

If the company’s executives are so certain that the 2024 marketing blitz worked, why are they pulling Apple Intelligence ads from YouTube when “nobody is looking”?

On another security note: knowing Apple’s penchant for user permission prompts (Shortcuts and macOS are the worst offenders), I wouldn’t be surprised if the company tried to mitigate Siri’s potential hallucinations and/or the risk of prompt injections with permission dialogs everywhere, and later realized the experience was terrible. Remember: Apple announced an App Intents-driven system with assistant schemas that included actions for your web browser, file manager, camera, and more. Getting any of those actions wrong (think: worse than not picking your mom up at the airport, but actually deleting some of your documents) could have pretty disastrous consequences.

Regardless of what happened, here’s the kicker: according to Mark Gurman, “some within Apple’s AI division” believe that the delayed Apple Intelligence features may be scrapped altogether and replaced by a new system rebuilt from scratch. From his story, pay close attention to this paragraph:

There are also concerns internally that fixing Siri will require having more powerful AI models run on Apple’s devices. That could strain the hardware, meaning Apple either has to reduce its set of features or make the models run more slowly on current or older devices. It would also require upping the hardware capabilities of future products to make the features run at full strength.

Inference costs may have gone down over the past 12 months and context windows may have gotten bigger, but I’m guessing there’s only so much you can do locally with 8 GB of RAM when you have to draw on the user’s personal context across (potentially) dozens of different apps, and then have conversations with the user about those results. It’ll be interesting to watch what Apple does here within the next 1-2 years: more RAM for the same price on iPhones, even more tasks handed off to Private Cloud Compute, or a combination of both?

We’ll see how this will play out at WWDC 2025 and beyond. I continue to think that Apple and Google have the most exciting takes on AI in terms of applying the technology to user’s phones and apps they use everyday. The only difference is that one company’s announcements were theoretical, and the other’s are shipping today. It seems clear now that Apple got caught off guard by LLMs while they were going down the Vision Pro path, and I’ll be curious to see how their marketing strategy will play out in the coming months.

Podcast Rewind: A Keyboard Rabbit-Hole, British Spies, and Converting Video to 3D

Enjoy the latest episodes from MacStories’ family of podcasts:

Comfort Zone

Niléane has entered her keyboard era, Chris has a consumer warning as well as a really good macropad, and then we find the best games on the Mac.

MacStories Unwind

This week, a little Cajun cooking, behind-the-scenes of the launch of NPC XL, another great British spy drama, and a classic sci-fi deal.

Magic Rays of Light

Sigmund and Devon grade their predictions for the 97th Academy Awards, discuss the process and benefits of converting films and shows to 3D, and recap the latest Apple Immersive Video release: Deep Water Solo.

Read more

Apple Delays Siri Personalization→

Apple released a statement to John Gruber of Daring Fireball today announcing that it is delaying a “more personalized Siri.” According to Apple’s Jacqueline Roy:

Siri helps our users find what they need and get things done quickly, and in just the past six months, we’ve made Siri more conversational, introduced new features like type to Siri and product knowledge, and added an integration with ChatGPT. We’ve also been working on a more personalized Siri, giving it more awareness of your personal context, as well as the ability to take action for you within and across your apps. It’s going to take us longer than we thought to deliver on these features and we anticipate rolling them out in the coming year.

This isn’t surprising given where things stand with Siri and Apple Intelligence more generally, but it is still disappointing. Of all the features shown off at WWDC last year, the ability to have Siri take actions in multiple apps on your behalf through natural language requests was one of the most eagerly anticipated. But, I’d prefer to get a feature that works than one that is half-baked.

Still, you have to wonder where the rest of the AI market will be by the time a “more personalized Siri” is released and whether it will look as much like yesterday’s tech as some of today’s Apple Intelligence features do.

Permalink

Phoenix Rising: Tapbots Reveals New Bluesky Client for Summer Release

Tapbots, the makers of Mastodon client Ivory, announced today that they are working on a Bluesky client. The app, which will be called Phoenix, is planned for release this summer.

There aren’t a lot of details yet, but Tapbots’ history of making top-notch social media apps stretches back many years, to Tweetbot, which was pulled from the App Store after Twitter no longer offered a third-party API. If you visit the project’s landing page, there is a brief FAQ, which explains that Ivory will continue to be developed, and that the Tapbots team believes separate apps for Mastodon and Bluesky will result in a better user experience than combining the two, which strikes me as right. Tapbots also says that they will release a public alpha version of the app as soon as possible.

This is great news. I’ve never been a huge fan of the default Bluesky app, and although more alternatives seem to be released every week, there are still fewer choices than there are for Mastodon. Plus, as an Ivory user, I’m looking forward to what I expect will be a similar interaction model and design with Phoenix.

Podcast Rewind: The Disappearing Prosumer App Market, Retroid Flip 2 Specs, NPC XL Debuts, and Tony Hawk Remakes Announced

Enjoy the latest episodes from MacStories’ family of podcasts:

AppStories

This week, Federico and I examine the widening gap between consumer and enterprise apps, asking what happened to prosumer and small business app market.

This episode is sponsored by:

  • Incogni – Take your personal data back with Incogni! Use code APPSTORIES with this link and get 60% off an annual plan.

NPC: Next Portable Console

This week, Retroid Pocket Flip 2 specs are out, the Legion Go S disappoints, and we try to decipher what AYANEO is doing before reminiscing over a PictoChat clone on the iPhone and discussing the benefits of having a personal videogame shopper in Japan, and Brendon’s experience with the black Miyoo Flip.

NPC XL

Then on NPC XL for Patreon backers, Federico, Brendon, and I share our love of the Sony PSP, from Federico’s long-time obsession to Brendon’s more recent journey down the rabbit hole, to my very first PSP, the gang covers the PSP’s history, their experiences, and the PSP’s impact on handheld gaming.

Back NPC on Patreon for a new episode of NPC XL every week.

Ruminate

Robb is hyped for a new Tony Hawk game, I launched NPC XL, and finally, Robb has a pen and paper update.

Read more