The Photos app has numerous Apple Intelligence features.
Collections
Collections is a mix of manually created items, metadata-sorted categories, pre-Apple Intelligence on-device machine learning selections, and new Apple Intelligence groupings. Click or tap Collections to see categories like Albums (created by you); Media Types (metadata); and People & Pets, Memories, and Trips (all machine learning). Featured Photos also appears, which is an album that adds new images daily from your library based on some kind of hidden algorithm about what Photos thinks you’d like best.
Collections includes a Utilities section, which is where you’ll find Apple Intelligence features.
Utilities
The Utilities section is a grab bag of things, including media that Photos can’t upload to iCloud, duplicated items, and recently saved ones. Apple Intelligence also makes use of this section by including categories it creates by analyzing your photo library.
You may not see all of these Utilities sets, depending on what’s in your Photos library.
Here are the Apple Intelligence-related groupings:
- Receipts: Photos of receipts as identified by Apple Intelligence appear here. On one of our devices, this category also showed a variety of other things, such as a junior-high report card, shipping labels, and even a plastic stake that came with a potted plant. The other saw letters photographed at library archives, shipping labels, and even pages from a book—but mostly receipts.
- Handwriting: Any photo containing what Apple Intelligence identifies as handwritten text is in this category. For one of us, this category was slightly more successful than Receipts, but not uniformly so. The other sees a fair number of images that had a handwriting font, and pictures of sushi and holiday cards.
- Identity Documents: Photos of passports, driver’s licenses (more formally, driver licenses), ID cards, debit and credit cards, and other documents that might serve to prove one’s identity appear here, with mixed success.
- Illustrations: Apple Intelligence apparently has a broad and eclectic view of what might be considered an illustration. Among other things, One of us saw photos of artwork from museums alongside drawings his kids made in first grade, graffiti, and even a furnace wiring diagram. The other, a printing and comics historian, found many printed cartoons and old line-drawing illustrations in his set.
- Documents: As with the Illustrations category, this shows anything vaguely document-like, as guessed by Apple Intelligence. For one of us, that included posters, menus, and signs—along with a carton of milk, a ketchup packet, and one of our kids holding a bag of chips. The other found the detritus of menus, screen captures, and pictures of his children being read books when young.
Generate reworked images
Apple offers three tools for taking a photo and generating new versions that add or remove information using generative imaging.
To use any of these tools, first select a photo and click Edit or tap the Edit button. Then click the Tools tab or tap the Tools button.
The first time you use any of these features, Photos must download the tool from Apple, which may take a few seconds to a few minutes. You should see a dialog that explains the delay.
All of these tools create new imagery based on existing data—you’re inventing parts of an image. That may be appropriate for personal images, or, once you consider it, you might not want to use any of these tools. Think about how the images will be received when shared, or if used in a professional, web, or print context, too.
Photos edited using artificial intelligence features (Clean Up, Extend, and Spatial Reframing) are invisibly tagged with a SynthID watermark.
Clean Up
The Clean Up tool in Photos lets you remove people and objects from the background of photos, automatically filling that space with new image information that blends seamlessly with the surroundings.
After selecting Clean Up, Photos may highlight certain areas of the photo with an animated, glowing effect. To remove one of those areas, click or tap it. You can also circle an object or brush over it with your pointer, finger, or Apple Pencil to remove it; the area may glow to indicate the selection. On a Mac, if the width of the Clean Up “brush” is too large or too small, use the Size slider.
Click or tap Clean Up to apply the changes. If and when you’re satisfied with the results: on a Mac, click Save Changes, then Done; on an iPhone or iPad, tap the yellow checkmark, then tap Done.
In this section, there’s also a Retouch brush that lets you smooth out details. However, it doesn’t use Apple Intelligence.
Reframe
If a photo was taken from an inauspicious angle, you can now adjust the angle—within limits—for a more pleasing effect, thanks to the Spatial Reframing feature of Apple Intelligence.
Select the Reframe tool. A glowing animation passes over the image. You can use your mouse or fingers to drag the image in any direction and watch as Photos dynamically fills in bits that were otherwise obscured. On a Mac, you can use Rotate, Pan, Straighten, and Zoom sliders, which also show the extent of the limits of how you can reframe the image. On an iPhone or iPad, use two fingers for zoom.
When you have the angle you want, click or tap Reframe.
Extend
Suppose a photo is cropped too narrowly, or that the subject is off-center. Photos can use Apple Intelligence to fill in areas adjacent to the area actually captured by generating new pixels that seamlessly blend in with what’s already in the photo. Select the Extend tool. Drag the frame to resize and/or reposition it and click or tap Extend. The results can be seemingly accurate or wildly disturbing.
Make Memories from prompts
You can make memories—rather, Memories with a capital M—using Apple Intelligence. Type a description, and Apple claims the feature will tell a coherent story sculpted from your photos and videos. To try this out, choose File > New Memory Movie, type a description, and wait to see what Photos comes up with.
One of us has had poor results with this feature, especially with music that bears no relation to the subject matter, but your experience may differ.
Use descriptive searches
Even if you never add a title or other metadata to an image in Photos, the app has long been able to figure out what (and who) is in it, so you could search for bridge or rainbow or dogs and usually get reasonable matches. Descriptive searching expands this concept, enabling you to type a whole phrase describing what you’re looking for, even if it involves a combination of attributes.
In our testing, this feature has been frighteningly accurate and helpful. We entered things like Morgen with the kids, Soren playing music, and California scenery and got results that meet those descriptions precisely.
