When an agent uses an app on your Mac, it can also use the app's menu bar, zoom into its screenshot and, if you allow it, use your clipboard.
- Menus. The agent can list an app's menus, with each item's shortcut and whether it is checked or dimmed, and press an item. This happens in the background, without opening a menu. It won't press a dimmed item.
- Zoom. The agent can look closer at part of its latest screenshot of an app, to read small text.
- Clipboard. The agent can read the text you copied or replace it, up to 10,000 characters at a time.
What is asked and what is refused:
- Menus and zoom follow the same per-app question as the rest of computer use (Allow an agent to control a specific app).
- Zoom works in terminals and code editors. Menus don't.
- The first time the agent wants your clipboard in a task, the app asks "Let the agent use your clipboard?". One answer covers reading and replacing for that task.
- The clipboard is closed to agents while a password, keychain or security app is in front.
Where: Mac app, in any chat that uses computer use.
Good to know: what a password manager copied is never shown to the agent, and text the agent puts on your clipboard is left out of the computer-use record. Related: What is computer use?, Apps an agent can only partly use, or not at all.