Let QwenWork operate supported apps on your computer.
Let QwenWork operate the apps on your computer directly
Beyond handling files and data in conversations, QwenWork can directly "see" and "operate" your computer screen — clicking buttons, filling in forms, and switching between apps. This capability is called Computer Control, and it lets QwenWork complete tasks that need to be performed in a graphical interface.
Screen awareness — Reads the visible content of the target app window and understands visual information such as layout, button text, and form states. During operation, it keeps taking screenshots to confirm whether the previous step succeeded before deciding the next action.
Mouse and keyboard control — Supports the full range of human input: clicking, double-clicking, dragging, typing, and keyboard shortcuts. Precision reaches the pixel level, so even small UI elements can be clicked accurately.
Autonomous background execution — Drives the mouse, keyboard, and screenshots in the background without taking your foreground focus. You can keep using your computer for other things while the AI quietly completes the task behind the scenes.
Cross-app workflows — Switches between multiple desktop apps and chains multi-step operations into a complete workflow. It adjusts the next step dynamically based on real-time feedback, rather than following a fixed script.
Simply describe the task you need completed in a graphical interface, and QwenWork will decide on its own when to enable Computer Control:
Before using Computer Control, enable the "Computer Control" connector in Connectors:
1. Open Connectors
Go to Extensions & Integrations → Connectors, find the "Computer Control" card, and click to enable it.
2. Review the permission notes
A dialog shows what this connector can do:
After enabling the connector, the first time you use Computer Control, QwenWork will guide you through granting two system permissions:
1. Accessibility
Allows QwenWork to read the UI element tree of apps and perform clicks, typing, and other actions. Simply allow it in the prompt from the system.
2. Screen Recording
Allows QwenWork to capture the screen content of app windows so the AI can "see" the current interface state. Add QwenWork under Privacy & Security → Screen Recording in System Settings.
Once permissions are granted, QwenWork will ask for your confirmation whenever the AI tries to operate an app. You can choose the execution policy in Settings:
When QwenWork performs Computer Control, you will see a fully transparent operation flow in the conversation:
Describe the target app and path clearly
Telling QwenWork which app to operate and where to go is much clearer than simply saying "set this up for me".
Break complex instructions into steps
If there are many steps, you can tell the AI in several rounds — complete the first step, confirm it is correct, then continue.
Combine with Skills for automation
Recurring GUI operation flows can be saved as a Skill, so the whole flow can be triggered with a single sentence later.
Granting access means granting control. Once enabled, the AI can drive other apps on your computer under your identity, with the same effect as if you operated them yourself. Disable it in Settings when you don't need it.
Core Capabilities
Screen awareness — Reads the visible content of the target app window and understands visual information such as layout, button text, and form states. During operation, it keeps taking screenshots to confirm whether the previous step succeeded before deciding the next action.
Mouse and keyboard control — Supports the full range of human input: clicking, double-clicking, dragging, typing, and keyboard shortcuts. Precision reaches the pixel level, so even small UI elements can be clicked accurately.
Autonomous background execution — Drives the mouse, keyboard, and screenshots in the background without taking your foreground focus. You can keep using your computer for other things while the AI quietly completes the task behind the scenes.
Cross-app workflows — Switches between multiple desktop apps and chains multi-step operations into a complete workflow. It adjusts the next step dynamically based on real-time feedback, rather than following a fixed script.
Use Cases
- Operating desktop apps without an API — When the target app has no command line or plugin, the AI operates the graphical interface directly. For example, adjusting parameters in a design tool or bulk-modifying settings in an admin console.
- Cross-app workflows — When a task spans multiple apps, the AI switches windows, copies data, and fills in forms automatically, completing the work end to end.
- GUI verification and testing — Confirm that interface changes work as expected, reproduce issues that only appear in the GUI, or check how an app responds to a specific sequence of operations.
- Collecting and consolidating information — Extract data from apps without an export feature, or aggregate information scattered across multiple apps.
System Requirements
- macOS 14 or later
- Accessibility and Screen Recording permissions must be granted
How to Use
Simply describe the task you need completed in a graphical interface, and QwenWork will decide on its own when to enable Computer Control:
Enabling Computer Control
Before using Computer Control, enable the "Computer Control" connector in Connectors:
1. Open Connectors
Go to Extensions & Integrations → Connectors, find the "Computer Control" card, and click to enable it.- AI enhancement tools — Provides desktop automation capabilities: clicking, typing, scrolling, taking screenshots, and other desktop control tools
- Always respects permissions — While Computer Control is running, it handles system permissions on its own as needed
- You stay in control — You can disable this connector from this dialog at any time
Granting System Permissions
After enabling the connector, the first time you use Computer Control, QwenWork will guide you through granting two system permissions:
1. Accessibility
Allows QwenWork to read the UI element tree of apps and perform clicks, typing, and other actions. Simply allow it in the prompt from the system.
2. Screen Recording
Allows QwenWork to capture the screen content of app windows so the AI can "see" the current interface state. Add QwenWork under Privacy & Security → Screen Recording in System Settings.
Once permissions are granted, QwenWork will ask for your confirmation whenever the AI tries to operate an app. You can choose the execution policy in Settings:
| Policy | Description |
|---|---|
| Ask Every Time (default) | The AI asks for your confirmation before operating the desktop each time |
| Auto-Execute | The AI performs desktop operations directly, without asking each time |
| Disabled | Turns off Computer Control completely |
How Operations Proceed
When QwenWork performs Computer Control, you will see a fully transparent operation flow in the conversation:
- Screenshot — The AI captures the current screen to understand the interface state
- Action description — Before each step, it explains what it is about to do
- Actual action — Performs the click, typing, scroll, or other action
- Result confirmation — After the action, it takes another screenshot to confirm the result matches expectations
Typical Scenarios
Extracting data from an app
Adjusting system settings
Consolidating information across apps
Usage Tips
Describe the target app and path clearly
Telling QwenWork which app to operate and where to go is much clearer than simply saying "set this up for me".
Break complex instructions into steps
If there are many steps, you can tell the AI in several rounds — complete the first step, confirm it is correct, then continue.
Combine with Skills for automation
Recurring GUI operation flows can be saved as a Skill, so the whole flow can be triggered with a single sentence later.
Cautions
Granting access means granting control. Once enabled, the AI can drive other apps on your computer under your identity, with the same effect as if you operated them yourself. Disable it in Settings when you don't need it.
- Some actions cannot be undone — Actions the AI performs in desktop apps (such as sending messages or deleting files) may be irreversible. For high-risk scenarios, we recommend the "Ask Every Time" policy.
- Screen content will be captured — The AI perceives the interface through screenshots, so anything visible on the screen (including sensitive information) may be captured. Close windows containing passwords or private data before running automation.
- Be careful with online actions — If the AI operates an app where you are signed in, it can send emails, submit forms, and more on your behalf. Stay vigilant about such actions.
Limitations
- CAPTCHAs and two-step verification — The AI cannot complete CAPTCHAs, SMS verification codes, face recognition, and similar steps; you need to step in manually.
- Speed — Computer Control requires taking screenshots and analyzing the screen, so it is slower than pure text operations.
- Precision — When the interface is complex or elements are densely packed, the AI's clicking accuracy may drop. If an operation fails, try describing it more specifically.