Apple Intelligence Features: All New AI Features Coming to iPhone, iPad & Mac

Oct. 28, 2024

1. Writing Tools

Writing Tools is an AI feature, powered by Apple Intelligence, and you can use this feature in any text field, on any app. This Apple Intelligence feature lets you rewrite texts, proofread paragraphs, change the tone, and more. You can also add your prompt to rewrite the text according to your preference or select a text to get a summary out of it. That’s not all. You cando a lot more with Writing Tools, like pull out key points from a long paragraph, make a list, or quickly turn data & numbers into a table. Writing Tools are available across iOS, iPadOS, and macOS. So, you can access these tools in Messages, Notes, Mail, and third-party apps as well.

Of all the new AI features coming to Apple devices, the most exciting is the more capable and more personal new Siri which is deeply integrated within the system. It has a new design and shows a glowing light around the corners when activated. The new Siri is pretty fast and can perform a wide range of on-device tasks. You can now ask questions about your iPhone, in case you can’t find a setting or a feature. It also has “on-screen awareness” and can see your screen to understand context and perform suitable actions.

Not only that, Apple Intelligence creates an on-device semantic index that stores all the relevant information. Whenever there is a user intent for an action, the relevant information is used to further process the user request. For example, if you ask the new Siri to check your flight schedule, it can automatically pull all your flight details from the Mail app and show you the current status. It can also set a schedule based on that information. It can perform many such on-device tasks which seems to be a gamechanger.

Apple has partnered with OpenAI to integrate ChatGPT across the system. With ChatGPT Integration, Apple’s AI system can handle on-device intelligence as well as complex queries that are beyond the capabilities of Siri.Apple has integrated ChatGPTinto iOS, iPadOS, and macOS, allowing users to access it via Siri, Writing Tools, and other first-party apps & features.

When a user asks a complicated question that Siri can’t handle, it will by default seek the user’s permission to hand off that query to ChatGPT. Then, the response will be relayed through Siri. ‌Besides, ChatGPT can also help compose text in Writing Tools. You can simply describe what you want, and ChatGPT will create it for you.

It’s worth knowing that ChatGPT Integration is separate from Apple Intelligence as your queries go to OpenAI’s cloud servers. However, Apple mentions that user queries won’t be logged by OpenAI.

The newAI-powered Clean Up toolis Apple’s equivalent to Google’s Magic Eraser. The Clean Up tool uses advanced detection and multiple machine learning models to identify and remove distractions in an image without affecting the overall picture. When you remove an element from the image, the Clean Up tool uses AI to generate a complementing background. The Clean Up tool appears as a new option (eraser-style icon in the bottom toolbar) when you edit an image in the Photos app. It works with all images in the ‌Photos‌ app, including older images and pictures captured by a DSLR camera or another phone.

Coming later this year, Image Want is an Image Playground-based feature that can transform your rough sketch into a related image in the Notes app. On your iPad, you can use Apple Pencil to draw a circle around your image, and Image Wand will analyze the content around it to whip a nice visual that complements your notes. When you circle an empty space, Image Wand will use the surrounding context to generate a complementing picture.

In the Mail app, Apple Intelligence puts time-sensitive messages on top of your inbox so that you don’t miss any deadlines. You can also read summaries of long emails in just one tap. If you want to quickly reply to an email or message, you can use Smart Reply to send a tailor-made response. Also, Multiple Messages or Mail notifications are summarized on your Lock Screen so you can see all the notifications without opening the app. Apple also plans to add a feature to the Mail app that will automatically sort your incoming messages into categories for better organization of things like newsletters and purchase emails.

Apple Intelligence can also prioritize notifications iniOS 18. It intelligently puts your important notifications on top of the stack and summarises them so you can pay attention at a glance. And if a message is important, it shows a ‘maybe important’ label on top of the notification stack, even when Focus mode is on.

Apple Intelligence also brings a super-useful feature to Safari that lets you summarize web pages. When you’re in reader mode, you’ll see a brand-new Summarize option toget the AI-generated summary of the webpage.This feature quickly summarizes text and is quite helpful in cases where you don’t want to scroll through large blocks of text.

Apple’s AI system introduces an all-new Focus Mode named ‘Reduce Interruptions’ that uses Apple Intelligence to surface only the important notifications while filtering out everything else. When youset up and use Reduce Interruptions Focus Mode, you’ll only see notifications that might need immediate attention. For instance, you’ll see notifications about an early pickup from playschool or your dental visit appointment confirmation. This is a useful Apple Intelligence feature that lets you stay on top of your important notifications without being carried away by unwanted distractions.

Believe it or not, Apple has broughtcall recording to iPhones with iOS 18. So, you can finallyrecord phone calls on your iPhone, without going for any workaround or a paid app. When you tap the Record button, it will relay an audio message that call recording has been enabled. The best part is that you also get the transcribe feature for recorded calls which saves call summaries so that you can quickly find important information later on. You can also transcribe audio files in the Notes app.

Visual Intelligence is a super-cool iPhone 16 exclusive feature that lets you usethe Camera Control buttonand on-device intelligence to discover anything that your camera points to. Imagine Apple’s version of Google Lens. For instance, when you point the camera to a restaurant, your iPhone will bring details like open hours, ratings, menu, and more.

WithiOS 18.1, Apple is rolling out the first serving of its AI features, including Writing Tools, Clean Up tool, new Siri UI, and more. Some of the most impressive features like Genmoji, ChatGPT integration, Image Playground, Image Wand, and more will arrive at a later date with future iOS 18.2, iOS 18.3, and iOS 18.4 updates.