Microsoft lets Copilot Studio use a computer on its own
- Microsoft has enabled a new “computer use” feature for Copilot Studio, allowing AI agents to interact with websites and desktop applications.
- This feature enables agents to perform tasks such as data entry, market research, and invoice processing, even when there is no API available to connect directly.
- The new tool can detect changes in buttons and screens on apps or websites and continue working without failing or breaking.
- Copilot Studio will be free to work with more websites and apps, unlike the consumer Copilot’s “Actions” feature which has limited partners.
- With this feature, businesses can build AI agents that treat websites and desktop apps as tools for automating tasks, similar to OpenAI’s Operator or Claude’s computer use feature.
Microsoft has enabled a new “computer use” feature for Copilot Studio this week that lets AI agents interact with websites and desktop applications. Much like OpenAI’s Operator or Claude’s identically named “computer use” feature, businesses will be able to use Copilot Studio to build AI agents that treat websites and desktop apps as if they’re tools for automating tasks.
“Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen,” explains Charles Lamanna, corporate vice president of Microsoft’s business & industry Copilot. “This allows agents to handle tasks even when there is no API available to connect to the system directly. If a person can use the app, the agent can too.”
Copilot Studio will be able to create AI agents that can automate data entry, perform market research, or even process invoices. This new tool will even detect when buttons and screens change in apps or websites and continue working without failing or breaking, according to Microsoft.
Microsoft added a similar feature, dubbed Actions, to its consumer Copilot earlier this month. Actions can be performed in the background while you work on other tasks. Copilot can now do things like book restaurant reservations, event tickets, and purchase items from online stores. The Actions experience is limited to a number of partners though, and it sounds like Copilot Studio will be free to work with more websites and apps.