Posts tagged with "artificial intelligence"

Apple Reveals A Partial Timeline for the Rollout of More Apple Intelligence Features

Last week, Apple released the first developer betas of iOS 18.2, iPadOS 18.2, and macOS 15.2, which the press speculated would be out by the end of the year. It turns out that was a good call because today, Apple confirmed that timing. In its press release about the Apple Intelligence features released today, Apple revealed that the next round is coming in December and will include the following:

  • Users will be able to describe changes they want made to text using Writing Tools. For example, you can have text rewritten with a certain tone or in the form of a poem.
  • ChatGPT will be available in Writing Tools and when using Siri.
  • Image Playground will allow users to create images with Apple’s generative AI model.
  • Users will be able to use prompts to create Genmoji, custom emoji-style images that can be sent to friends in iMessage and used as stickers.
  • Visual intelligence will be available via the Camera Control on the iPhone 16 and iPhone 16 Pro. The feature will allow users to point the iPhone’s camera at something and learn about it from Google or ChatGPT. Apple also mentions that visual intelligence will work with other unspecified “third-party tools.”
  • Apple Intelligence will be available in localized English in Australia, Canada, Ireland, New Zealand, South Africa, and the U.K.

Apple’s press release also explains when other languages are coming:

…in April, a software update will deliver expanded language support, with more coming throughout the year. Chinese, English (India), English (Singapore), French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, and other languages will be supported.

And Apple’s Newsroom in Ireland offers information on the Apple Intelligence rollout in the EU:

Mac users in the EU can access Apple Intelligence in U.S. English with macOS Sequoia 15.1. This April, Apple Intelligence features will start to roll out to iPhone and iPad users in the EU. This will include many of the core features of Apple Intelligence, including Writing Tools, Genmoji, a redesigned Siri with richer language understanding, ChatGPT integration, and more.

It’s a shame it’s going to be another six months before EU customers can take advantage of Apple Intelligence features on their iPhones and iPads, but it’s nonetheless good to hear when it will happen.

It’s also worth noting that the timing of other pieces of Apple Intelligence is unclear. There is still no word on precisely when Siri will gain knowledge of your personal context or perform actions in apps on your behalf, for instance. Even so, today’s reveal is more than Apple usually shares, which is both nice and a sign of the importance the company places on these features.

Apple’s Commitment to AI Is Clear, But Its Execution Is Uneven

The day has finally arrived. iOS 18.1, iPadOS 18.1, and macOS 15.1 are all out and include Apple’s first major foray into the world of artificial intelligence. Of course, Apple is no stranger to AI and machine learning, but it became the narrative that the company was behind on AI because it didn’t market any of its OS features as such. Nor did it have anything resembling the generative AI tools from OpenAI, Midjourney, or a host of other companies.

However, with today’s OS updates, that has begun to change. Each update released today includes a far deeper set of new features than any other ‘.1’ release I can remember. Not only are the releases stuffed with a suite of artificial intelligence tools that Apple collectively refers to as Apple Intelligence, but there are a bunch of other new features that Niléane has written about, too.

The company is tackling AI in a unique and very Apple way that goes beyond just the marketing name the features have been given. As users have come to expect, Apple is taking an integrated approach. You don’t have to use a chatbot to do everything from proofreading text to summarizing articles; instead, Apple Intelligence is sprinkled throughout Apple’s OSes and system apps in ways that make them convenient to use with existing workflows.

If you don't want to use Apple Intelligence, you can turn it off with a single toggle in each OS's settings.

If you don’t want to use Apple Intelligence, you can turn it off with a single toggle in each OS’s settings.

Apple also recognizes that not everyone is a fan of AI tools, so they’re just as easy to ignore or turn off completely from System Settings on a Mac or Settings on an iPhone or iPad. Users are in control of the experience and their data, which is refreshing since that’s far from given in the broader AI industry.

The Apple Intelligence features themselves are a decidedly mixed bag, though. Some I like, but others don’t work very well or aren’t especially useful. To be fair, Apple has said that Apple Intelligence is a beta feature. This isn’t the first time that the company has given a feature the “beta” label even after it’s been released widely and is no longer part of the official developer or public beta programs. However, it’s still an unusual move and seems to reveal the pressure Apple is under to demonstrate its AI bona fides. Whatever the reasons behind the release, there’s no escaping the fact that most of the Apple Intelligence features we see today feel unfinished and unpolished, while others remain months away from release.

Still, it’s very early days for Apple Intelligence. These features will eventually graduate from betas to final products, and along the way, I expect they’ll improve. They may not be perfect, but what is certain from the extent of today’s releases and what has already been previewed in the developer beta of iOS 18.2, iPadOS 18.2, and macOS 15.2 is that Apple Intelligence is going to be a major component of Apple’s OSes going forward, so let’s look at what’s available today, what works, and what needs more attention.

Read more

New Developer Betas Released for iOS, iPadOS, and macOS with Image Playground, ChatGPT Integration, and More Apple Intelligence Features

iOS 18.1, iPadOS 18.1, and macOS 15.1 aren’t quite out the door, but Apple has already updated its developer betas with the next round of upcoming Apple Intelligence features. Developer betas of iOS 18.2, iPadOS 18.2, and macOS 15.2 are now available for download and include the following:

  • image generation in the form of Image Playground and Image Wand;
  • Genmoji (iOS and iPadOS only)
  • Visual Intelligence (iPhone 16 line only)
  • ChatGPT integration with Siri; and
  • new text manipulation features.
Image Playground. Source: Apple.

Image Playground. Source: Apple.

Image Playground is a feature that allows you to create images in two styles using in-app themes and other tools. Image Playground is available in apps like Messages, Freeform, Pages, and Keynote, but it’s also a standalone app. Regardless of where you use it, Image Playground looks like it’s designed to make it easy to create animated and sketch-style images using a variety of tools such as suggested concepts that pull from the context the image is created in, like a Messages thread. Creations can be previewed, there’s a history feature that allows you to undo changes made to images, and images are saved to an Image Playground Library that syncs across devices via iCloud.

Image Wand. Source: Apple.

Image Wand. Source: Apple.

Image Wand, which appears in the Apple Pencil tool palette, takes a rough hand-drawn sketch, photo, or note and turns any of them into an image similar to one created by Image Playground. Image Wand can be further refined by adding text, and if you circle a blank space, it will use surrounding text to build an image.

Also, Genmoji – which is only in the iOS and iPadOS betas for now – allows you to create emoji-style images that can be used in Messages and other apps as decorative stickers. Inputs can include a text description, people in your contacts, friends and family recognized in Photos, and characters created from whole cloth.

Visual Intelligence has been added to the Camera Control on the iPhone 16 line too. The feature lets you look up details about a place and work with text, copying, reading, summarizing, and translating it.

The next betas also integrate ChatGPT into Siri. As demoed at WWDC, you can opt to pose queries to ChatGPT without disclosing you identity or IP address and without the prompts being used to train OpenAI’s large language models. The ChatGPT integration is free and does not require an account with OpenAI either.

Writing Tools lets you describe your text changes in iOS 18.2, iPadOS 18.2, and macOS 15.2.

Writing Tools lets you describe your text changes in iOS 18.2, iPadOS 18.2, and macOS 15.2.

Finally, Apple has built a new Writing Tool that provides additional flexibility when manipulating text. From the Writing Tools UI, you’ll be able to submit a prompt to alter any text you’ve written. For instance, you could have Apple Intelligence make you sound more excited in your message or rewrite it in the form of a poem, neither of which is possible with the Writing Tools found in iOS and iPadOS 18.1 or macOS 15.1.

For developers, there are also new APIs for Writing Tools, Genmoji, and Image Playground.

As we’ve covered before, Apple’s AI models have been trained on a mix of licensed data and content from the web. If you’re a publisher or a creator who doesn’t want to be part of those models, you can opt out, but it doesn’t work retroactively. In other words, opting out won’t remove any data already ingested by Apple’s web crawlers, but it will work going forward.

I’m not a fan of generative AI tools, but I am looking forward to finally going beyond tightly controlled demos of these features. I want to see how well they work in practice and compare them to other AI tools. Apple appears to have put a lot of guardrails in place to avoid some of the disasters that have befallen other tech companies, but I’m pretty good at breaking software. It will be interesting to see how well these tools hold up under pressure.

Craig Federighi on Apple’s Private Cloud Compute Architecture→

Apple’s Craig Federighi was interviewed by Wired about the company’s Private Cloud Compute infrastructure that will handle Apple Intelligence requests that can’t be handled locally on-device. Federighi told Wired’s Lily Hay Newman:

What was really unique about the problem of doing large language model inference in the cloud was that the data had to at some level be readable by the server so it could perform the inference. And yet, we needed to make sure that that processing was hermetically sealed inside of a privacy bubble with your phone. So we had to do something new there. The technique of end-to-end encryption—where the server knows nothing—wasn’t possible here, so we had to come up with another solution to achieve a similar level of security.

Still, Apple says that it offers “end-to-end encryption from the user’s device to the validated PCC nodes, ensuring the request cannot be accessed in transit by anything outside those highly protected PCC nodes.” The system is architected so Apple Intelligence data is cryptographically unavailable to standard data center services like load balancers and logging devices. Inside a PCC cluster, data is decrypted and processed, but Apple emphasizes that once a response is encrypted and sent on its journey to the user, no data is retained or logged and none of it is ever accessible to Apple or its individual employees.

PCC is a complex system that leverages technologies that Apple has developed like the Secure Enclave, Secure Boot, and Trusted Execution Monitor to ensure customer privacy. Those technologies are backstopped by making every PCC server publicly available for inspection and verification by third parties. That said, Wired spoke to security researchers and cryptography experts who told the publication that although PCC looks promising, they hadn’t spent significant time studying it yet.

Cloud-based privacy is a tough problem to solve. I’m keen to hear what independent researchers think of Apple’s solution, once they’ve had more time to evaluate it. If PCC is as robust as Apple claims, it’s the sort of thing I could see Apple turning into a standalone product as worldwide regulation pushes tech companies to offer better privacy protections for their customers.

Permalink

Procreate Will Not Include Generative AI in Its Apps→

Today on its website, Procreate announced that it would not build generative AI tools into its apps. The company’s position is clear and unequivocal:

Generative AI is ripping the humanity out of things. Built on a foundation of theft, the technology is steering us toward a barren future. We think machine learning is a compelling technology with a lot of merit, but the path generative AI is on is wrong for us.

We’re here for the humans. We’re not chasing a technology that is a moral threat to our greatest jewel: human creativity. In this technological rush, this might make us an exception or seem at risk of being left behind. But we see this road less travelled as the more exciting and fruitful one for our community.

In a short video on X.com, Procreate CEO James Cuda said:

I don’t like what’s happening to the industry, and I don’t like what it’s doing to artists.

I couldn’t agree more or be happier to see Procreate take a stand in defense of artists. Federico and I interviewed Cuda at WWDC, and although Cuda struck a diplomatic tone having just received an Apple Design Award for Procreate Dreams, it was clear to me then that we were unlikely to see generative AI in Procreate’s apps. For everyone who wasn’t in the room with Cuda that day, though, today’s statement should set their minds at ease. I hope we see more developers whose apps support creative fields take a similar stand.

Permalink

Apple Says It Won’t Ship Major New OS Features in the EU This Fall Due to DMA Uncertainty→

A new round in the fight between the EU and Apple has been brewing for a while now. About a week ago, the Financial Times reported that unnamed sources said that the EU was poised to levy significant fines against the company over a probe of Apple’s compliance with the Digital Markets Act. Then, earlier this week, in an interview with CNBC, the EU’s competition chief, Margrethe Vestager telegraphed that Apple is facing enforcement measures:

[Apple] are very important because a lot of good business happens through the App Store, happens through payment mechanisms, so of course, even though you know I can say this is not what was expected of such a company, of course we will enforce exactly with the same top priority as with any other business.

Asked when enforcement might happen, Vestager told CNBC ‘hopefully soon.’

Apple made no comment to CNBC at the time, but today, that shoe has apparently dropped, with Apple telling the Financial Times that:

Due to the regulatory uncertainties brought about by the Digital Markets Act, we do not believe that we will be able to roll out three of these [new] features – iPhone Mirroring, SharePlay Screen Sharing enhancements, and Apple Intelligence – to our EU users this year.

Is it a coincidence that Apple made its statement to the same media outlet that reported that fines were about to be assessed? I doubt it. The more likely scenario is that Apple is using OS updates as a negotiating chip with EU regulators. Your guess is as good as mine whether the move will work. Personally, I think the tactic is just as likely to backfire. However, I’m quite confident that you’ll be hearing from me again about fines by the EU against Apple sooner rather than later.

Permalink

Wired Confirms Perplexity Is Bypassing Efforts by Websites to Block Its Web Crawler

Last week, Federico and I asked Robb Knight to do what he could to block web crawlers deployed by artificial intelligence companies from scraping MacStories. Robb had already updated his own site’s robots.txt file months ago, so that’s the first thing he did for MacStories.

However, robots.txt only works if a company’s web crawler is set up to respect the file. As I wrote earlier this week, a better solution is to block them on your server, which Robb did on his personal site and wrote about late last week. The setup sends a 403 error if one of the bots listed in his server code requests information from his site.

Spoiler: Robb hit the nail on the head the first time.

Spoiler: Robb hit the nail on the head the first time.

After reading Robb’s post, Federico and I asked him to do the same for MacStories, which he did last Saturday. Once it was set up, Federico began testing the setup. OpenAI returned an error as expected, but Perplexity’s bot was still able to reach MacStories, which shouldn’t have been the case.1

Yes, I took a screenshot of Perplexity's API documentation because I bet it changes based on what we discovered.

Yes, I took a screenshot of Perplexity’s API documentation because I bet it changes based on what we discovered.

That began a deep dive to try to figure out what was going on. Robb’s code checked out, blocking the user agent specified in Perplexity’s own API documentation. What we discovered after more testing was that Perplexity was hitting MacStories’ server without using the user agent it said it used, effectively doing an end run around Robb’s server code.

Robb wrote up his findings on his website, which promptly shot to the top slot on Hacker News and caught the eye of Dhruv Mehrotra and Tim Marchman of Wired, who were in the midst of investigating how Perplexity works. As Mehrotra and Marchman describe it:

A WIRED analysis and one carried out by developer Robb Knight suggest that Perplexity is able to achieve this partly through apparently ignoring a widely accepted web standard known as the Robots Exclusion Protocol to surreptitiously scrape areas of websites that operators do not want accessed by bots, despite claiming that it won’t. WIRED observed a machine tied to Perplexity—more specifically, one on an Amazon server and almost certainly operated by Perplexity—doing this on wired.com and across other Condé Nast publications.

Until earlier this week, Perplexity published in its documentation a link to a list of the IP addresses its crawlers use—an apparent effort to be transparent. However, in some cases, as both WIRED and Knight were able to demonstrate, it appears to be accessing and scraping websites from which coders have attempted to block its crawler, called Perplexity Bot, using at least one unpublicized IP address. The company has since removed references to its public IP pool from its documentation.

That secret IP address—44.221.181.252—has hit properties at Condé Nast, the media company that owns WIRED, at least 822 times in the last three months. One senior engineer at Condé Nast, who asked not to be named because he wants to “stay out of it,” calls this a “massive undercount” because the company only retains a fraction of its network logs.

WIRED verified that the IP address in question is almost certainly linked to Perplexity by creating a new website and monitoring its server logs. Immediately after a WIRED reporter prompted the Perplexity chatbot to summarize the website’s content, the server logged that the IP address visited the site. This same IP address was first observed by Knight during a similar test.

This sort of unethical behavior is why we took the steps we did to block the use of MacStories’ websites as training data for Perplexity and other companies.2 Incidents like this and the lack of transparency about how AI companies train their models have led to a lot of mistrust in the entire industry among creators who publish on the web. I’m glad we’ve been able to play a small part in revealing Perplexity’s egregious behavior, but more needs to be done to rein in this sort of behavior, including closer scrutiny by regulators around the world.

As a footnote to this, it’s worth noting that Wired also puts to rest the argument that websites should be okay with Perplexity’s behavior because they include citations in their plagiarism. According to Wired’s story:

WIRED’s own records show that Perplexity sent 1,265 referrals to wired.com in May, an insignificant amount in the context of the site’s overall traffic. The article to which the most traffic was referred got 17 views.

That’s next to nothing for a site with Wired’s traffic, which Similarweb and other sites peg at over 20 million page views that same month. That’s a mere 0.006% of Wired’s May traffic. Let that sink in, and then ask yourself whether it seems like a fair trade.

  1. Meanwhile, I was digging through bins of old videogames and hardware at a Retro Gaming Festival doing ‘research’ for NPC. ↩︎
  2. Mehrotra and Marchman correctly question whether Perplexity is even an AI company because they piggyback on other company’s LLMs and use them in conjunction with scraped web data to provide summaries that effectively replace the source’s content. However, that doesn’t change the fact that Perplexity is surreptitiously scraping sites while simultaneously professing to respect sites’ robot.txt file. That’s the unethical bit. ↩︎

Apple Developer Academies in Six Countries to Add AI Courses This Fall

Today, Apple announced that this fall, the company will offer a new curriculum for its Developer Academy students focused on machine learning and artificial intelligence.

According to Apple:

Beginning this fall, every Apple Developer Academy student will benefit from custom-built curriculum that teaches them how to build, train, and deploy machine learning models across Apple devices. Courses will include the fundamentals of AI technologies and frameworks; Core ML and its ability to deliver fast performance on Apple devices; and guidance on how to build and train AI models from the ground up. Students will learn from guided curriculum and project-based assignments that include assistance from hundreds of mentors and more than 12,000 academy alumni worldwide.

The new curriculum will be offered at 18 academies in Brazil, Indonesia, Italy, Saudi Arabia, South Korea, and the United States. With the company’s emphasis on Apple Intelligence at WWDC, it’s not surprising that the skills needed to implement those new features are being added to its educational efforts.

How We’re Trying to Protect MacStories from AI Bots and Web Crawlers – And How You Can, Too

Over the past several days, we’ve made some changes at MacStories to address the ingestion of our work by web crawlers operated by artificial intelligence companies. We’ve learned a lot, so we thought we’d share what we’ve done in case anyone else would like to do something similar.

If you read MacStories regularly, or listen to our podcasts, you already know that Federico and I think that crawling the Open Web to train large language models is unethical. Industry-wide, AI companies have scraped the content of websites like ours, using it as the raw material for their chatbots and other commercial products without the consent or compensation of publishers and other creators.

Now that the horse is out of the barn, some of those companies are respecting publishers’ robots.txt files, while others seemingly aren’t. That doesn’t make up for the tens of thousands of articles and images that have already been scraped from MacStories. Nor is robots.txt a complete solution, so it’s just one of four approaches we’re taking to protect our work.

Read more