The Vision OS for the Apple Vision Pro Headset includes a feature called “Visual Search” that looks similar to the iPhone and iPad’s Visual Search.
With visual search, users can use the Vision Pro headset to get information about an item, detect and interact with text in the world around them, copy and paste printed text from the real world into apps, translate text between 17 different languages, and more.
Real-world text that includes contact information, web pages, unit conversions, and similar information can be handled in visionOS. So, for example, if a print flyer contains a website link, you can scan the link with Vision Pro, and open a Safari window to view the website. Or, if the recipe calls for grams and you need ounces, you can convert with your headset.
Real-time text translation will also be useful for travel and other situations where you might want to quickly translate what you see in the real world. The Apple Vision Pro headset will be able to automatically detect text and documents, similar to how the iPhone can detect text in photos and allow interaction with it.
Visual search function found in visionOS by Steve Moser. visionOS can be accessed through the latest Xcode beta right now, as Apple released the first version of the software earlier today.
