The Hermes computer use skill is the part of Hermes Agent that lets the AI work your desktop apps for you, clicking, typing and scrolling in the background while you carry on with your own work.
To be precise, Hermes does this with a built-in tool called computer_use, and a bundled skill called Computer Use teaches the model how to use that tool properly.
Hermes Agent is the free, open-source agent from Nous Research, and both the tool and the skill come with it.
For a business, the interesting part is that the agent can use the same apps your team uses, such as an email client or a file manager, without taking over your mouse.
This guide explains what it is, how to switch it on and seven real jobs it can handle, using Nous's official documentation as the source.
My video below covers the newest computer-use update, where the agent's screen can move onto a throwaway Cua computer.
It's a news walkthrough using Cua's own demo footage, so I'm not claiming I've tested that new setup myself.
Hermes Computer Use Skill Jobs At A Glance
These are the seven jobs I cover below, with the feature that makes each one possible.
| Job | The feature that makes it work | Why a business cares |
|---|---|---|
| Reading and acting in desktop apps | The computer_use tool with the bundled skill. |
The agent can use apps that have no API. |
| Checking a machine from your phone | Real screenshots sent on Telegram, Discord or Desktop. | You see the actual screen, not just a description. |
| Repeatable jobs on a schedule | Bounded mode with a reviewed capability manifest. | You approve the scope once and the job stays inside it. |
| A bot on a server with its own screen | Bot Screen on a Linux gateway. | The bot keeps working after you close your laptop. |
| Working beside you on one machine | Background input with no focus steal. | You keep typing while the agent clicks elsewhere. |
| Several agents at once | A separate session cursor for each run and subagent. | You can tell which agent is doing what. |
| Client work on a throwaway computer | The cua-spaces plugin with a Linux Cua Space. | The agent's clicks never touch your own desktop. |
🔥 Want my exact Hermes Computer Use Skill setup, step by step? Inside the AI Profit Boardroom you get step-by-step video trainings, my installable Agent OS and four coaching calls a week with 3,400+ members building real automations. → Join the AI Profit Boardroom
What The Hermes Computer Use Skill Is, In Plain English
Think of the tool as the agent's hands and the skill as its training manual.
The computer_use tool takes screenshots, reads the list of buttons and fields on screen, and clicks or types.
The Computer Use skill, version 2.1.0, tells the model to capture the screen first, click by numbered element, check the result and follow strict safety rules.
The skill page lists its authors as Francesco Bonacci (f-trycua) and Hermes Agent, and Francesco is the founder and CTO of Cua.
That's because the tool runs on cua-driver, an open-source driver from Cua's trycua/cua project.
It works on macOS, Windows and Linux, and the docs say it works with any tool-capable model, including Claude, GPT, Gemini and local models.
The Problem It Solves For A Business
Here's the old way.
A lot of business software has no API, so automating it meant clunky scripts that grabbed your mouse and broke whenever a window moved.
You couldn't use your computer while the script ran, because it was literally moving your cursor.
The new way is an agent that works in the background.
The docs say your cursor doesn't move, your keyboard focus doesn't change and your virtual desktops don't switch.
You and the agent co-work on the same machine, and the agent reads the screen like a person would.
How To Switch It On
The docs say the driver ships with Hermes, so most people only need to switch the toolset on.
hermes computer-use status
hermes -t computer_use chat
You can also run hermes tools and pick Computer Use, which installs the driver if it's missing.
On macOS, grant Accessibility and Screen Recording to CuaDriver, and hermes computer-use permissions grant asks macOS for both.
On Linux, you need a display server, and on Windows nothing extra is needed at install time.
If anything looks wrong, run hermes computer-use doctor, which prints a health check for every part of the setup.
The full operator walkthrough is in the five-step Hermes computer use skill setup on our sister site.
Job 1: Read And Act Inside Desktop Apps
This is the core job.
The docs' own example prompt is "Find my latest email from Stripe and summarise what they want me to do."
The agent captures the mail app, clicks the search field, types from:stripe, opens the top result and writes you a summary.
The skill says to reach for computer use when a task needs real native apps, such as a file manager, an email client, a native chat client or Figma.
That's where it earns its keep for a business, because those apps often have no easy way to automate them.
For web pages, the skill says to use Hermes's separate browser tools, because they are more reliable for page content.
Job 2: Check A Machine From Your Phone
When you're away from your desk, you can ask the agent on Telegram or Discord to "Send me a screenshot of my screen."
The docs say the agent delivers the real image as a native attachment, not just a description.
On the CLI there's no attachment channel, so the agent gives you the saved file's path.
Screenshots are never sent automatically, only when you ask, and only the 20 most recent captures are kept.
That's a simple way to check a long-running job without opening a laptop.
Job 3: Run Repeatable Jobs Inside A Fixed Scope
Some business jobs repeat, such as scheduled research against an app you're logged in to.
For those, the docs describe bounded mode, where you write a capability manifest once and approve it at launch.
computer_use:
permission_mode: bounded
capability_manifest: ~/.hermes/cua-manifest.yaml
The manifest names the apps, browser profile kinds, allowed origins and tools the session may use.
Anything outside the manifest fails closed inside the driver.
That's the setting I'd want before letting any agent touch a business account on a schedule.
🔥 Want to see how I use agents like this in a real business? Inside the AI Profit Boardroom, I've got Hermes Agent training, the Agent OS and step-by-step automation workflows. You'll be learning with 3,400+ members, and there are four coaching calls every week where you can ask your questions. → Get access here
Job 4: A Bot On A Server With Its Own Screen
A server has no monitor, so computer use needs somewhere to look.
Bot Screen, added in v0.22.0, gives each bot on a Linux gateway its own Xfce desktop that Hermes Desktop streams live.
The docs say the bot keeps working after you close the app or turn off your laptop, because the screen lives on the gateway host.
When the bot hits a login, 2FA prompt, CAPTCHA or payment step, it says so and ends its turn.
You click Take over, do that step yourself, click Hand back, and the bot carries on with the session you just signed in to.
Every Hermes profile has its own screen, browser profile and cookies, so each bot keeps its own logins.
The docs plan on about 1.1 to 1.5 GB of memory per open screen with a browser, so size your server with that in mind.
If you're new to Hermes bots, read my Hermes Agent bot mode guide first.
Job 5: Work Beside You On One Machine
Because input runs in the background, you can keep typing in your editor while the agent works in another window.
The skill's background rules tell the model not to raise windows, not to switch virtual desktops and not to pop anything to the front.
On platforms where it's enabled, a tinted overlay cursor shows where the agent is clicking, while your real cursor stays put.
The docs say Hermes turns that overlay off by default on macOS and Linux X11, and keeps it on Windows and Linux Wayland.
Job 6: Run Several Agents At Once
Each Hermes run declares its own session name, such as hermes-3a7b9c14d2e8.
The docs say concurrent runs and subagents get distinct cursors, so you can see which agent is acting where.
For an agency running several jobs, that makes supervision much less confusing.
Job 7: Do Client Work On A Throwaway Computer
This is the update from my video at the top.
Cua's cua-spaces plugin moves Hermes's terminal into a Cua Space, which is an isolated machine you can watch live.
On Hermes 0.22.3 and later, the plugin catalog says Bot Desktop, computer_use and the browser follow the terminal into the Space.
That works in Linux Spaces, with the cua command on your PATH and the desktop stack installed in the Space.
The business win is that the agent clicks around on a disposable computer rather than the desktop where your email and banking are open.
The full setup is in my Hermes Cua guide, and the wider safety options are in my Hermes Agent sandbox guide.
If you want a second opinion on where desktop agents fit your agency, you can book a free SEO strategy session with my team.
The Safety Rules That Protect Your Business
- Clicks, typing, drags, scrolls and key presses go through the same approval gate as dangerous shell commands.
- Empty trash, force delete, lock screen and log out are hard-blocked.
- Typed text that matches dangerous patterns such as
sudo rm -rf /is blocked. - Where nobody can approve, such as a cron job or a headless run, actions are refused rather than auto-approved.
- The skill tells the model never to type passwords, API keys or card numbers, and never to follow instructions found in screenshots.
The docs warn that YOLO or unrestricted mode doesn't protect against prompt injection, so keep it for disposable machines.
When Not To Use It
The skill itself lists the jobs where computer use is the wrong tool.
- Web automation is better done with the headless
browsertools, which the skill calls more reliable for pages. - File edits are better done with the file tools than by typing into an editor window.
- Shell commands are better done with the
terminaltool than by typing into a terminal app.
The Limits A Business Should Know About
- Some apps expose a sparse accessibility tree, so the agent falls back to clicking by position.
- A normal Windows agent can't drive elevated admin windows.
- macOS support relies on private system interfaces that Apple can change in any update.
- Screenshots cost tokens, although the docs say a 20-action session typically uses about 30,000 tokens of screenshot context.
🔥 Want my Hermes playbooks for client work? The AI Profit Boardroom has the Agent OS, Hermes Agent training and four weekly coaching calls where you can ask me how to roll desktop agents out safely. → Join the AI Profit Boardroom
Related Reading
- Seven ways a business uses Hermes Cua
- Hermes Agent bot mode for business
- How a business uses a Hermes Agent sandbox
Also On Our Network
- 🌐 Switch on the Hermes computer use skill in five steps
- 🌐 Ten Hermes computer use features, scored
- 🌐 The Hermes computer use skill inside an Agent OS
- 🌐 The models that drive Hermes computer use best, on GoldieBench
FAQ: Hermes Computer Use Skill For Business
What can the Hermes computer use skill do for a business?
It lets Hermes operate desktop apps such as email clients and file managers in the background.
The docs' examples include summarising your latest Stripe email and sending you a real screenshot of a screen.
Is the Hermes computer use skill free?
Hermes, the tool and the skill are free and open source.
You still pay for the AI model you connect, unless you run a local one.
Will the agent take over my mouse while I work?
No, it works in the background, so your cursor and keyboard focus stay where they are.
Can the agent log in to accounts for me?
The skill tells the model never to type passwords or handle 2FA or payment prompts.
With Bot Screen, you take over the screen for the login and then hand back.
Is it safe to let an agent use my computer?
Hermes routes input through approvals and hard-blocks risky actions.
For more safety, run the agent's screen inside a sandbox or a Cua Space.
If you run an agency and want help planning agent work for your clients, you can book a free strategy session and we'll map it out together.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link-building agency, Goldie Agency, with 70+ people in Bangkok.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote the book "Link Building Mastery", which covers nine link building strategies.
- I run GoldieBench, where I score AI models on real one-shot builds.
→ Get my best AI training inside the AI Profit Boardroom
Pick one desktop job, switch on the toolset, keep the safety rails on, and that's how a business starts getting real work out of the Hermes computer use skill.
