Posts tagged with "artificial intelligence"

New Developer Betas Released for iOS, iPadOS, and macOS with Image Playground, ChatGPT Integration, and More Apple Intelligence Features

iOS 18.1, iPadOS 18.1, and macOS 15.1 aren’t quite out the door, but Apple has already updated its developer betas with the next round of upcoming Apple Intelligence features. Developer betas of iOS 18.2, iPadOS 18.2, and macOS 15.2 are now available for download and include the following:

  • image generation in the form of Image Playground and Image Wand;
  • Genmoji (iOS and iPadOS only)
  • Visual Intelligence (iPhone 16 line only)
  • ChatGPT integration with Siri; and
  • new text manipulation features.
Image Playground. Source: Apple.

Image Playground. Source: Apple.

Image Playground is a feature that allows you to create images in two styles using in-app themes and other tools. Image Playground is available in apps like Messages, Freeform, Pages, and Keynote, but it’s also a standalone app. Regardless of where you use it, Image Playground looks like it’s designed to make it easy to create animated and sketch-style images using a variety of tools such as suggested concepts that pull from the context the image is created in, like a Messages thread. Creations can be previewed, there’s a history feature that allows you to undo changes made to images, and images are saved to an Image Playground Library that syncs across devices via iCloud.

Image Wand. Source: Apple.

Image Wand. Source: Apple.

Image Wand, which appears in the Apple Pencil tool palette, takes a rough hand-drawn sketch, photo, or note and turns any of them into an image similar to one created by Image Playground. Image Wand can be further refined by adding text, and if you circle a blank space, it will use surrounding text to build an image.

Also, Genmoji – which is only in the iOS and iPadOS betas for now – allows you to create emoji-style images that can be used in Messages and other apps as decorative stickers. Inputs can include a text description, people in your contacts, friends and family recognized in Photos, and characters created from whole cloth.

Visual Intelligence has been added to the Camera Control on the iPhone 16 line too. The feature lets you look up details about a place and work with text, copying, reading, summarizing, and translating it.

The next betas also integrate ChatGPT into Siri. As demoed at WWDC, you can opt to pose queries to ChatGPT without disclosing you identity or IP address and without the prompts being used to train OpenAI’s large language models. The ChatGPT integration is free and does not require an account with OpenAI either.

Writing Tools lets you describe your text changes in iOS 18.2, iPadOS 18.2, and macOS 15.2.

Writing Tools lets you describe your text changes in iOS 18.2, iPadOS 18.2, and macOS 15.2.

Finally, Apple has built a new Writing Tool that provides additional flexibility when manipulating text. From the Writing Tools UI, you’ll be able to submit a prompt to alter any text you’ve written. For instance, you could have Apple Intelligence make you sound more excited in your message or rewrite it in the form of a poem, neither of which is possible with the Writing Tools found in iOS and iPadOS 18.1 or macOS 15.1.

For developers, there are also new APIs for Writing Tools, Genmoji, and Image Playground.

As we’ve covered before, Apple’s AI models have been trained on a mix of licensed data and content from the web. If you’re a publisher or a creator who doesn’t want to be part of those models, you can opt out, but it doesn’t work retroactively. In other words, opting out won’t remove any data already ingested by Apple’s web crawlers, but it will work going forward.

I’m not a fan of generative AI tools, but I am looking forward to finally going beyond tightly controlled demos of these features. I want to see how well they work in practice and compare them to other AI tools. Apple appears to have put a lot of guardrails in place to avoid some of the disasters that have befallen other tech companies, but I’m pretty good at breaking software. It will be interesting to see how well these tools hold up under pressure.

Craig Federighi on Apple’s Private Cloud Compute Architecture→

Apple’s Craig Federighi was interviewed by Wired about the company’s Private Cloud Compute infrastructure that will handle Apple Intelligence requests that can’t be handled locally on-device. Federighi told Wired’s Lily Hay Newman:

What was really unique about the problem of doing large language model inference in the cloud was that the data had to at some level be readable by the server so it could perform the inference. And yet, we needed to make sure that that processing was hermetically sealed inside of a privacy bubble with your phone. So we had to do something new there. The technique of end-to-end encryption—where the server knows nothing—wasn’t possible here, so we had to come up with another solution to achieve a similar level of security.

Still, Apple says that it offers “end-to-end encryption from the user’s device to the validated PCC nodes, ensuring the request cannot be accessed in transit by anything outside those highly protected PCC nodes.” The system is architected so Apple Intelligence data is cryptographically unavailable to standard data center services like load balancers and logging devices. Inside a PCC cluster, data is decrypted and processed, but Apple emphasizes that once a response is encrypted and sent on its journey to the user, no data is retained or logged and none of it is ever accessible to Apple or its individual employees.

PCC is a complex system that leverages technologies that Apple has developed like the Secure Enclave, Secure Boot, and Trusted Execution Monitor to ensure customer privacy. Those technologies are backstopped by making every PCC server publicly available for inspection and verification by third parties. That said, Wired spoke to security researchers and cryptography experts who told the publication that although PCC looks promising, they hadn’t spent significant time studying it yet.

Cloud-based privacy is a tough problem to solve. I’m keen to hear what independent researchers think of Apple’s solution, once they’ve had more time to evaluate it. If PCC is as robust as Apple claims, it’s the sort of thing I could see Apple turning into a standalone product as worldwide regulation pushes tech companies to offer better privacy protections for their customers.

Permalink

Procreate Will Not Include Generative AI in Its Apps→

Today on its website, Procreate announced that it would not build generative AI tools into its apps. The company’s position is clear and unequivocal:

Generative AI is ripping the humanity out of things. Built on a foundation of theft, the technology is steering us toward a barren future. We think machine learning is a compelling technology with a lot of merit, but the path generative AI is on is wrong for us.

We’re here for the humans. We’re not chasing a technology that is a moral threat to our greatest jewel: human creativity. In this technological rush, this might make us an exception or seem at risk of being left behind. But we see this road less travelled as the more exciting and fruitful one for our community.

In a short video on X.com, Procreate CEO James Cuda said:

I don’t like what’s happening to the industry, and I don’t like what it’s doing to artists.

I couldn’t agree more or be happier to see Procreate take a stand in defense of artists. Federico and I interviewed Cuda at WWDC, and although Cuda struck a diplomatic tone having just received an Apple Design Award for Procreate Dreams, it was clear to me then that we were unlikely to see generative AI in Procreate’s apps. For everyone who wasn’t in the room with Cuda that day, though, today’s statement should set their minds at ease. I hope we see more developers whose apps support creative fields take a similar stand.

Permalink

Apple Says It Won’t Ship Major New OS Features in the EU This Fall Due to DMA Uncertainty→

A new round in the fight between the EU and Apple has been brewing for a while now. About a week ago, the Financial Times reported that unnamed sources said that the EU was poised to levy significant fines against the company over a probe of Apple’s compliance with the Digital Markets Act. Then, earlier this week, in an interview with CNBC, the EU’s competition chief, Margrethe Vestager telegraphed that Apple is facing enforcement measures:

[Apple] are very important because a lot of good business happens through the App Store, happens through payment mechanisms, so of course, even though you know I can say this is not what was expected of such a company, of course we will enforce exactly with the same top priority as with any other business.

Asked when enforcement might happen, Vestager told CNBC ‘hopefully soon.’

Apple made no comment to CNBC at the time, but today, that shoe has apparently dropped, with Apple telling the Financial Times that:

Due to the regulatory uncertainties brought about by the Digital Markets Act, we do not believe that we will be able to roll out three of these [new] features – iPhone Mirroring, SharePlay Screen Sharing enhancements, and Apple Intelligence – to our EU users this year.

Is it a coincidence that Apple made its statement to the same media outlet that reported that fines were about to be assessed? I doubt it. The more likely scenario is that Apple is using OS updates as a negotiating chip with EU regulators. Your guess is as good as mine whether the move will work. Personally, I think the tactic is just as likely to backfire. However, I’m quite confident that you’ll be hearing from me again about fines by the EU against Apple sooner rather than later.

Permalink

Wired Confirms Perplexity Is Bypassing Efforts by Websites to Block Its Web Crawler

Last week, Federico and I asked Robb Knight to do what he could to block web crawlers deployed by artificial intelligence companies from scraping MacStories. Robb had already updated his own site’s robots.txt file months ago, so that’s the first thing he did for MacStories.

However, robots.txt only works if a company’s web crawler is set up to respect the file. As I wrote earlier this week, a better solution is to block them on your server, which Robb did on his personal site and wrote about late last week. The setup sends a 403 error if one of the bots listed in his server code requests information from his site.

Spoiler: Robb hit the nail on the head the first time.

Spoiler: Robb hit the nail on the head the first time.

After reading Robb’s post, Federico and I asked him to do the same for MacStories, which he did last Saturday. Once it was set up, Federico began testing the setup. OpenAI returned an error as expected, but Perplexity’s bot was still able to reach MacStories, which shouldn’t have been the case.1

Yes, I took a screenshot of Perplexity's API documentation because I bet it changes based on what we discovered.

Yes, I took a screenshot of Perplexity’s API documentation because I bet it changes based on what we discovered.

That began a deep dive to try to figure out what was going on. Robb’s code checked out, blocking the user agent specified in Perplexity’s own API documentation. What we discovered after more testing was that Perplexity was hitting MacStories’ server without using the user agent it said it used, effectively doing an end run around Robb’s server code.

Robb wrote up his findings on his website, which promptly shot to the top slot on Hacker News and caught the eye of Dhruv Mehrotra and Tim Marchman of Wired, who were in the midst of investigating how Perplexity works. As Mehrotra and Marchman describe it:

A WIRED analysis and one carried out by developer Robb Knight suggest that Perplexity is able to achieve this partly through apparently ignoring a widely accepted web standard known as the Robots Exclusion Protocol to surreptitiously scrape areas of websites that operators do not want accessed by bots, despite claiming that it won’t. WIRED observed a machine tied to Perplexity—more specifically, one on an Amazon server and almost certainly operated by Perplexity—doing this on wired.com and across other Condé Nast publications.

Until earlier this week, Perplexity published in its documentation a link to a list of the IP addresses its crawlers use—an apparent effort to be transparent. However, in some cases, as both WIRED and Knight were able to demonstrate, it appears to be accessing and scraping websites from which coders have attempted to block its crawler, called Perplexity Bot, using at least one unpublicized IP address. The company has since removed references to its public IP pool from its documentation.

That secret IP address—44.221.181.252—has hit properties at Condé Nast, the media company that owns WIRED, at least 822 times in the last three months. One senior engineer at Condé Nast, who asked not to be named because he wants to “stay out of it,” calls this a “massive undercount” because the company only retains a fraction of its network logs.

WIRED verified that the IP address in question is almost certainly linked to Perplexity by creating a new website and monitoring its server logs. Immediately after a WIRED reporter prompted the Perplexity chatbot to summarize the website’s content, the server logged that the IP address visited the site. This same IP address was first observed by Knight during a similar test.

This sort of unethical behavior is why we took the steps we did to block the use of MacStories’ websites as training data for Perplexity and other companies.2 Incidents like this and the lack of transparency about how AI companies train their models have led to a lot of mistrust in the entire industry among creators who publish on the web. I’m glad we’ve been able to play a small part in revealing Perplexity’s egregious behavior, but more needs to be done to rein in this sort of behavior, including closer scrutiny by regulators around the world.

As a footnote to this, it’s worth noting that Wired also puts to rest the argument that websites should be okay with Perplexity’s behavior because they include citations in their plagiarism. According to Wired’s story:

WIRED’s own records show that Perplexity sent 1,265 referrals to wired.com in May, an insignificant amount in the context of the site’s overall traffic. The article to which the most traffic was referred got 17 views.

That’s next to nothing for a site with Wired’s traffic, which Similarweb and other sites peg at over 20 million page views that same month. That’s a mere 0.006% of Wired’s May traffic. Let that sink in, and then ask yourself whether it seems like a fair trade.

  1. Meanwhile, I was digging through bins of old videogames and hardware at a Retro Gaming Festival doing ‘research’ for NPC. ↩︎
  2. Mehrotra and Marchman correctly question whether Perplexity is even an AI company because they piggyback on other company’s LLMs and use them in conjunction with scraped web data to provide summaries that effectively replace the source’s content. However, that doesn’t change the fact that Perplexity is surreptitiously scraping sites while simultaneously professing to respect sites’ robot.txt file. That’s the unethical bit. ↩︎

Apple Developer Academies in Six Countries to Add AI Courses This Fall

Today, Apple announced that this fall, the company will offer a new curriculum for its Developer Academy students focused on machine learning and artificial intelligence.

According to Apple:

Beginning this fall, every Apple Developer Academy student will benefit from custom-built curriculum that teaches them how to build, train, and deploy machine learning models across Apple devices. Courses will include the fundamentals of AI technologies and frameworks; Core ML and its ability to deliver fast performance on Apple devices; and guidance on how to build and train AI models from the ground up. Students will learn from guided curriculum and project-based assignments that include assistance from hundreds of mentors and more than 12,000 academy alumni worldwide.

The new curriculum will be offered at 18 academies in Brazil, Indonesia, Italy, Saudi Arabia, South Korea, and the United States. With the company’s emphasis on Apple Intelligence at WWDC, it’s not surprising that the skills needed to implement those new features are being added to its educational efforts.

How We’re Trying to Protect MacStories from AI Bots and Web Crawlers – And How You Can, Too

Over the past several days, we’ve made some changes at MacStories to address the ingestion of our work by web crawlers operated by artificial intelligence companies. We’ve learned a lot, so we thought we’d share what we’ve done in case anyone else would like to do something similar.

If you read MacStories regularly, or listen to our podcasts, you already know that Federico and I think that crawling the Open Web to train large language models is unethical. Industry-wide, AI companies have scraped the content of websites like ours, using it as the raw material for their chatbots and other commercial products without the consent or compensation of publishers and other creators.

Now that the horse is out of the barn, some of those companies are respecting publishers’ robots.txt files, while others seemingly aren’t. That doesn’t make up for the tens of thousands of articles and images that have already been scraped from MacStories. Nor is robots.txt a complete solution, so it’s just one of four approaches we’re taking to protect our work.

Read more

AppStories, Episode 378 – Are We Entering a Post-App World?

This week on AppStories, we explore whether we’re experiencing the beginning of the end of apps and consider what might replace them.

Sponsored by:

  • Genius Scan – A scanner in your pocket.
  • Jam – Developer friendly bug reports in 1 click.

Are We Entering a Post-App World?

[Subscribe here.](https://appstories.net/plans)

Subscribe here.

On AppStories+, we explain why we’ve said goodbye to time tracking.

We deliver AppStories+ to subscribers with bonus content, ad-free, and at a high bitrate early every week.

To learn more about the benefits included with an AppStories+ subscription, visit our Plans page, or read the AppStories+ FAQ.

Read more

Club MacStories Sample: BetterTouchTool Tips, Vision Pro Shortcuts, a Task Manager Review, and the Effect of AI on the Internet

We often describe Club MacStories as more of the MacStories you know and love reading on this website. That’s an apt shorthand for the Club, but when you’re being asked to sign up and pay for something, it still helps to see what you’re buying. That’s why every now and then, we like to share samples of some of what the Club has to offer every week.

So today, we’ve made Issue 408 of MacStories Weekly from a couple of Saturdays ago available to everyone. Just use this link, and you’ll get the whole issue. You can also use the links in the excerpts below to read particular articles.

Everybody in the Club gets MacStories Weekly and our monthly newsletter called the Monthly Log, but there’s a lot more to the Club than just email newsletters. All members also get MacStories Unwind+, an ad-free version of the podcast that we publish a day early for Club members. All Club members also have access to a growing collection of downloadable perks like wallpapers and eBooks.

Club MacStories+ members get all of those perks along with exclusive columns that are published outside our newsletters, access to our Discord community, discounts on dozens of iOS, iPadOS, and Mac apps, and advanced search, filtering, and custom RSS feed creation of Club content. Club Premier builds on the first two tiers by adding AppStories+, the extended, ad-free version of our flagship podcast that’s delivered a day early, as well as full-text search of AppStories show notes, making it the all-access pass for everything we do at MacStories.

To learn more and sign up, you can use the buttons below:

Join Club MacStories:

Join Club MacStories+:

Join Club Premier:

Issue 408 of MacStories Weekly, which you can access here, starts with two excellent tips from Niléane on how to use BetterTouchTool to remap the Mac’s yellow and green ‘stoplight’ buttons. Like a lot of tips and workflows we share, Niléane’s was inspired by a similar technique Federico employed a couple of weeks before:

Two weeks ago, in Issue 406 of MacStories Weekly, Federico shared a tip for BetterTouchTool that resonated with me. Just like him, I am used to minimizing my windows instead of hiding them, which can be annoying since minimized windows no longer come up when you Command (⌘) + Tab to their app’s icon…

…after poking around in BetterTouchTool for a few minutes, I realized that the app allows you to change what the red, yellow, and green window buttons do. As a result, I was able to make it so that the yellow button will actually hide a window instead of minimizing it to the Dock.

Read more