Apple released the first developer beta version of iOS 18.2, with the most eye-catching feature being the integration of ChatGPT into Siri. This allows Siri to understand and answer complex questions about the user about what is displayed on the device's screen, such as identifying places or buildings in images. This feature is implemented by Siri taking screenshots and uploading them to ChatGPT for analysis, and completing the entire process after obtaining user consent. This update is especially important for iPhone users who do not have Visual Intelligence capabilities, as it extends the ability to understand the screen content.
Apple has released the first developer beta version of iOS 18.2, which includes many new Apple Intelligence features. One of the new features is the long-awaited integration with ChatGPT, which allows you to now ask Siri for details about what you see on the device’s screen.
By integrating ChatGPT into Siri, Apple's virtual assistant will use the OpenAI platform to answer complex questions. However, it can also read content on the iPhone, iPad, or Mac screen. For example, if you are watching a video or viewing someone’s photos on Instagram, you can ask questions like “Where is this?” or “What is the building on that photo?”
Source: 9to5mac
Siri takes screenshots and uploads them to ChatGPT, which is able to understand the context and provide answers using its database. Of course, Siri always asks for your permission before taking a screenshot of the device and sending it to ChatGPT for privacy reasons.
This feature is especially interesting for those without an iPhone 16, as the iPhone 16 is by far the only device to get Visual Intelligence. With this feature, users can press "Camera Control" to get information about what they see on the camera. Visual Intelligence uses Apple Intelligence, ChatGPT, and Google to provide answers.
Apple is also using the Apple Intelligence language model to develop its own screen-aware features. However, the feature will be launched with another update to iOS18 — possibly next year. Once the feature is available, Siri will understand the context and act on the content on the screen—all processing will be done on the device.
Source: 9to5mac
In addition to ChatGPT integration, iOS18.2 comes with Image Playground, Genmoji and a redesigned mail app.
Currently, iOS18.2 beta is only available for devices compatible with Apple Intelligence. This means iPhone15Pro or later and M1iPad or later. Of course, this update will eventually work for other devices as well.
Apple has previously hinted that iOS 18.2 will be released to the public in December. iOS 18.1 will be released next week and will feature Apple's smart writing tools along with notifications and text summary.
In short, the update of iOS 18.2 brings a smarter and more convenient experience to users, especially the integration with ChatGPT, which further enhances Siri's features, allowing it to better understand and process screen content information. We look forward to the release of the official version of iOS 18.2 to experience more new features.