Configuration
Computer Use
Computer Use lets agents observe and control applications on the host machine: listing windows, taking screenshots, clicking, typing, and running host shell commands, all gated by per-agent permissions.
Platform availability
Computer Use is available on macOS and Windows in the Electron desktop app only. It is not available in web server deployments or on Linux; the Settings > Computer Use tab shows a notice on unsupported platforms.
How it works
Gamut uses the @skillful-agents/agent-computer SDK, which communicates with a local daemon (ac-core) on the host machine. The daemon provides accessibility-level access to the OS, allowing agents to:
- Observe: List running applications, enumerate windows, take screenshots, read UI element trees, and query display information.
- Interact: Click elements, type text, fill form fields, press keyboard shortcuts, scroll, hover, select dropdown values, and work with menus and dialogs.
- Manage apps: Launch, relaunch, quit, grab (focus), and ungrab applications.
When an agent interacts with an application, a visual halo appears around the target window to signal that AI-driven control is active. The halo disappears when the agent releases (ungrabs) the window or is stopped.
Permission levels
Computer Use permissions come in three levels, from least to most powerful:
List Apps & Windows
Read-only access. The agent can list running applications, enumerate open windows, check system status, and query display information. This level does not allow interaction with any application.
Covers these operations: apps, windows, status, displays, permissions.
Use Application
App-specific interaction. The agent can interact with a specific named application: clicking, typing, taking screenshots, reading UI trees, and more. The permission is scoped to a single app; the agent must request permission for each app it wants to control.
Covers all interaction operations: click, type, fill, key, scroll, select, hover, snapshot, find, screenshot, read, launch, relaunch, quit, grab, ungrab, menuClick, dialog.
Host Shell
Shell command execution. The agent can run shell commands and scripts on the host machine with your user permissions. This is the most powerful level.
Permission grant types
When an agent requests a permission it does not have, a prompt appears in the UI to approve or deny the request. Pick a grant type at that point:
| Grant type | Duration | Persistence |
|---|---|---|
| Once | Single use; consumed immediately after the operation completes. | In-memory only. |
| Timed | 15 minutes from the time of grant. | In-memory only; lost on restart. |
| Always | Permanent until explicitly revoked. | Saved to settings.json and survives restarts. |
Managing permissions
Viewing active permissions
Open Settings > Computer Use to see all persistent ("Always") permissions, grouped by agent:
- The agent's name.
- Each granted permission level and, for "Use Application" grants, the specific app name.
Once and timed grants are transient, stored only in memory, so they do not appear here.
Revoking permissions
Permissions can be revoked at two levels of granularity:
- Revoke a single grant: Click the trash icon next to a specific permission entry.
- Revoke all grants for an agent: Click Revoke All on the agent's permission card.
Revocation takes effect immediately. If the agent attempts the operation again, a new grant prompt appears.
Automatic cleanup
When an agent container stops (manually or by auto-sleep), any active window grab is released and the visual halo disappears. Timed grants expire naturally after 15 minutes.
macOS permissions
On macOS, Computer Use requires two system-level permissions for the Gamut application:
- Accessibility: Required for reading UI element trees, clicking, typing, and other interaction operations.
- Screen Recording: Required for taking screenshots.
Gamut checks both via the ac-core daemon; if either is missing, Computer Use requests fail. Grant them in System Settings > Privacy & Security.
Security considerations
Computer Use gives agents significant power over the machine:
- Review each request. The permission prompt shows exactly what the agent is asking to do and which application it targets. Do not grant blanket "Always" permissions unless the agent's system prompt and behavior are trusted.
- Prefer scoped grants. "Use Application" permissions cover a single app. An agent allowed to use Safari cannot touch the Terminal without a separate grant.
- Use timed grants for temporary access. When an agent needs short-lived access (debugging a UI issue, for example), a 15-minute timed grant is safer than a permanent one.
- Host Shell is the most sensitive level. An agent with Host Shell can run arbitrary commands with your user privileges. Grant it only to agents you fully trust and have reviewed.
- Agents operate with your user permissions. Anything an agent does through Computer Use, you could do at your keyboard. There is no privilege escalation, but also no sandboxing beyond the permission system.
- Stop the agent to revoke all transient access. Stopping an agent releases any grabbed windows and clears all in-memory (once and timed) grants.