Anthropic has launched a new artificial intelligence capability called “computer use” that enables its AI to interact with users’ computer screens.
The feature, released in beta to developers on October 22, 2024, represents a significant advancement in AI assistance technology, moving beyond simple chatbot interactions to more sophisticated computer operations.
The new tool allows Anthropic’s AI to interpret on-screen content and, with user permission, perform various tasks such as web browsing, button clicking and text input. This development marks a shift in AI assistance technology, as it can process real-time screen activity rather than relying on back-end application integrations.
During a demonstration, the system reportedly showcased its abilities by planning a morning hike near the Golden Gate Bridge, autonomously searching for trails, checking sunrise times and creating calendar invites with detailed information about appropriate attire.
The release comes amid growing industry interest in AI agents that can operate with minimal human oversight. While companies like Microsoft and Salesforce have recently introduced their agent tools for workplace tasks, Anthropic’s approach differs in that it focuses on direct screen interpretation rather than application-specific integration.
More...
More Info