Microsoft Rolling Out Copilot Vision in Preview n2g2z Can Understand ’s Browsing Activity

Copilot Vision is currently available in preview to select Copilot Pro subscribers in the US. 726p3b

Written by Siddharth Suvarna | Updated: 6 December 2024 18:53 IST

Microsoft Rolling Out Copilot Vision in Preview, Can Understand ’s Browsing Activity

Photo Credit: Microsoft 3a34u

Microsoft says after the end of a browsing session, the information shared with Copilot is deleted

Highlights

Microsoft says Copilot Vision will work when using the Edge browser
Copilot Vision s voice-based prompts
Microsoft has kept Copilot Vision opt-in by default

ment

Microsoft introduced a new artificial intelligence (AI) capability for Copilot on Thursday. Dubbed Copilot Vision, it now enables the AI chatbot to see and understand the context of what a does online. The Redmond-based tech giant said that Copilot can access the Microsoft Edge browser. The vision-enabled chatbot can now monitor which websites and web pages the browses, and then can perform certain commands when prompted. This capability is currently available in preview to a select number of Copilot Pro subscribers in the US.

Microsoft Copilot Vision 635z4p

The Windows maker has been trying to bring its AI-powered chatbot Copilot to the forefront of the PC experience. Earlier, it unveiled purpose-built AI agents to complete tasks on the PC, and now, it has added another capability to Copilot that will let it answer queries based on their browser-related activity, as per the blog post. Notably, Copilot Vision is not an agentic function, so the AI cannot take actions on behalf of the .

Copilot Vision can understand the full context of what a sees on the Edge browser. It can see the images and elements, read the text, and even comprehend the navigation buttons. The vision-enabled chatbot will be present at the bottom of the screen and s can either type or speak the prompt.

Microsoft Finally Releases Controversial AI Recall Feature in Preview

Coming to what the AI can do, it can answer questions, summarise and translate text, and spotlight certain parts of the page when prompted to do so. Microsoft states that Copilot Vision can also act as a game assistant and help a understand how to play games like GeoGuessr, and offer tips while playing a game of Chess online.

Highlighting examples, the tech giant said Copilot Vision can help s plan an outing at the museum by pointing out the important information a should know. It can also help s with shopping by recommending the right products based on the 's needs and preferences.

On privacy and security, Microsoft stated that Copilot Vision will be entirely opt-in and s can decide when to turn it on and when to keep it turned off. The AI will not be able to see something that the does not want to. Additionally, any conversation data with the AI will be deleted at the end of each session. However, the Copilot's responses are logged and they will be used to improve the company's safety systems.

Microsoft’s AI Agents Will Soon Complete Complex Tasks Automatically

Microsoft further added that Copilot Vision does not capture, store, or use any data from publishers to train its AI models. It is also working with a group of external testers (red teamers) to improve the safety of the new feature.

NASA and Microsoft Launch AI Tool to Make Earth Data Accessible

Copilot Vision is currently rolling out to a limited number of Copilot Pro subscribers in the US and it can be accessed via Copilot Labs. Notably, Vision will currently work only on a select set of websites, and the functionality will be expanded gradually.

links may be automatically generated - see our ethics statement for details.

Comments

For the latest reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Microsoft Edge

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen ing his favourite football club - Chelsea, watching movies and anime, and sharing ionate opinions on food. More