Well, it couldn’t be helped for too long. Apple has just held its WWDC 2024 keynote live stream, which kicks off the week of WWDC at Cupertino, where the company helps developers from all over the world get versed in its platforms. Saved for the very end of the stream were Apple’s AI announcements, ushering in Apple’s entrance into the generative AI space, and giving us a guide as to what the company will be doing with the tech. Everyone, meet Apple Intelligence.
Apple Intelligence is the iPhone maker’s fancy name for artificial intelligence. It’s a blanket term for covering the rollout of AI features across the Apple product suite, with such features to come to MacOS, iOS, and iPadOS (nothing for TvOS and WatchOS for now).
And that blanket term ranges from features you’ll find in Mail to a complete Siri overhaul. Here are all the AI (Apple Intelligence, don’t wear it out) features coming to an Apple device near you soon.
Apple’s AI features, all in one place
Siri’s facelift
Let’s start with the Siri overhaul. In tandem with the launch of generative AI features across Apple’s devices, the company’s virtual assistant Siri is getting a bit of a refresh. This involves changing the visuals of it. Now, when you trigger Siri to start listening, a coloured ring will appear around the perimeter of your iPhone, though Apple hasn’t shown what this will entail for Mac-prompted Siri just yet, but it’s easy to expect the same for iPad Siri).
Apple claims that the AI-powered changes make Siri more natural, relevant, and personal to the user. There’s richer language processing at play with the upcoming changes; for example, Siri will understand the conversational context of what you’re asking, based on your last input. So, if you ask about a beach, and then in a second prompt, ask Siri to plan a hike there for the next day, it’ll understand, without needing you to say the name of the beach again.
Siri is also getting any time typing support, so you can ask Siri to do things without needing to use your voice. Perfect feature, thank you Apple. The virtual assistant will also be getting on-screen awareness, so it can process the text shown on the screen without you needing to tell it. New in-app features are also coming powered by Siri, so you can ask the assistant to perform certain actions, such as touch up a photo, or find something in your files.
Siri’s also getting a deeper understanding of your personal context, and will be able to find on-device information you’re looking for easier. Think stuff like a list your friend sent you, but you can’t remember which app it was in, or if you need to enter your licence information, and don’t have your card with you, but have it in photos – Siri’s got it covered.
Apparently Siri can also cross-reference current flight ETAs with times indicated by friends and family over text. That’s super cool.
System-wide writing tools
Across consumer-level AI applications, you’re probably well versed in generative AI’s ability to rewrite your words in different tones. Well, Apple’s getting that very feature.
Across system-wide writing apples, including Notes and Mail, along with third-party apps, you’ll be able to select bodies of text and select ‘rewrite’. This will automatically allow you to shift the tone to Friendly, Professional, or Concise. You’ll also be able to change the format; to a summary, to key points, to a table, or to a list. If you don’t want such a systemic rewrite, you can instead select ‘Proofread’, which will instead offer you suggestions for writing improvement (RIP Grammarly).
There’s also a new little feature called ‘Smart reply’, which takes the space of Suggestions above your keyboard when writing, offering to pen out an entire response to a text or an email. This can even be expanded into toggles, so you can draft a response based on basic facts (for example, answering if you’re taking an Uber or driving, or if your significant other is coming or not).
AI summaries and Priority Notifications
Summaries are also a pretty neat feature coming, offering a summarised explainer for emails in your inbox without showing the topline words.
Depending on urgency, Apple Intelligence will surface emails and messages to the tops of your inboxes and your notification stack (such as flight information or dinner reservations).
A new focus mode is also being added called ‘Reduce Interruptions’, an AI-powered mode that enables Do Not Disturb, but lets through sufficiently important messages, such as texts about upcoming events. This is kind of already a thing in Apple’s focus modes but I’m interested to see how its different.
Image Playground
Apple is adding a new app that’ll allow you to play around with a server-based image generator, called Image Playground. These images can then be saved and used elsewhere, and with Image Playground functionality being opened up as an API for developers, you can expect it to feed into other apps. No word from Apple on what model it used to train the image generator, though.
if anyone ever sends me one of these it’s an instant block pic.twitter.com/eP7neZqf4f
— → (@R3H8T) June 10, 2024
Image Playground will be integrated with Apple apps like Pages and Freeform, to allow you quick access to the generator. Apps like Notes will even allow you to circle illustrations you’ve done, and spruce them up with AI, or completely replace them.
One of the leading features is unique to Messages…
Genmoji
That’s right. Apple’s adding generative emojis called Genmoji, which take the emoji art style that Apple employs and allows you to go ahead and make your own. Just type in something like ‘T-Rex wearing a tutu on a surfboard’, and voila, you’ve got a neat party trick. You can even have the image generator make an image of someone in your contacts or photos.
Clean up and Photos features
Remember years ago when Google added Magic Eraser to its Pixel range of devices, allowing you to circle objects to completely remove them from a photo? Well, Apple’s doing that now, with a new feature called Clean Up, which acts similarly, allowing you to circle something and erase it, such as somebody in the background taking a selfie.
Searching for photos and videos in your Photos app is now easier. You can now type a more detailed search, such as for somebody wearing a specific colour of clothing, and the app will show such photos. You can even search for specific parts in videos based on a prompt, and create a ‘memory movie’, based on contextually relevant photos.
Notes and call transcribing
Notes is getting a transcription feature, allowing you to take down notes while recording what someone is saying, including converting the audio to text. You can even then generate a summary of what they were saying. Phone calls will also be getting a record and transcription feature, with the other participant being notified when you hit the record button.
Yes, ChatGPT on Apple devices
Apple lauded ChatGPT as the best of the AI models, the ‘Pioneer and market leader’, according to senior vice president of software engineering Craig Federighi, and now, the GPT will be getting integration with Apple devices. You won’t need to login to an OpenAI account, though users can do this to access deeper ChatGPT features if they’re a subscriber.
Siri’s got ChatGPT support, such as if you need cooking ideas (though it will ask for your permission to access ChatGPT or not). ChatGPT’s response will then be shown before you. You can also uploaded photos with prompts, such as if you need decorating advice, and show a shot of a balcony in need of plants.
Oh, and of course, the system-wide writing tools are getting a smattering of ChatGPT. The new ‘Compose’ feature will generate text based on a prompt, such as a bedtime story. You can also use the Compose feature to generate images from ChatGPT (separate to Image Playground).
Apple said that it intends to add support for other AI models in the future.
What devices will be getting Apple Intelligence?
Not every device will be getting Apple Intelligence, mind you. Apple said that it will be coming to iPhone 15 Pro, Pro Max, iPads, and Macs with M1 and later chips. It’ll be available in US English within the coming months as part of a beta for iOS 18.
No AI features were announced for TVOS, WatchOS, or VisionOS.
You can read about everything announced at WWDC 2024 here.
Screenshot: Gizmodo Australia