Hermes can now use your Mac. For free. In the background.

Hermes just got computer use — free, open-source, in the background

Nous Research dropped Computer Use for Hermes a few hours ago. It's a feature inside the free open-source Hermes agent that lets the model operate your Mac like a person would. Click. Type. Scroll. Drag. All in the background, while you're focused on something else entirely.

The killer detail: your cursor doesn't move. Your keyboard focus stays where it is. The agent doesn't yank you out of the window you're working in. You and Hermes are two hands on the same keyboard, both writing at once.

Works with any AI model with vision — Claude, ChatGPT, Gemini, or even a free local model via LM Studio. The only models it can't use are text-only ones, because the agent needs to see screenshots to know where to click.

This is a different category from what came before. OpenClaw → moved your real cursor around. Hermes Computer Use doesn't — it runs invisibly. Less dramatic to watch, but way more practical to use.

Why background is the whole game

Earlier computer-use agents had a tax: they hijacked your mouse. Watching the cursor fly around the screen looked impressive on demos, but in real work it meant you couldn't use your own computer while the agent was running. You were locked out for minutes at a time.

Hermes Computer Use changes the model. The agent operates through an invisible layer — it sees the screen via screenshots, decides what to do, then dispatches clicks and keystrokes without touching your active cursor or window focus.

You can journal a note in Notes while Hermes drafts another note in a different app. You can write in Obsidian while Hermes reorganises a different folder. Same machine. Different jobs. Parallel.

The invisible hand. No cursor on your screen. Just a trail of work appearing in apps Hermes is operating in the background.

The Goldie Shadow Worker™ — give a goal, oversee, transform

Most people will treat Computer Use as a toy. "Open my Notes app." Then nothing else. That's not where the leverage is. Here's the three-step framework I use:

Five-minute install

The whole setup takes about five minutes. Two commands, one permissions toggle.

• Update Hermes first. Run hermes update in your terminal to pull the latest agent + skills.
• Install Computer Use. Run hermes-computer-use install. This wires the MCP server and the skill into Hermes. Takes about 30 seconds.
• Grant OS permissions. Open System Settings → Privacy & Security → Accessibility, and allow Terminal. Hermes itself can read the same instructions back to you and walk you through it if you paste them into a chat.
• Test it. In Hermes, type: "Use computer use to list the open apps and reply with a short confirmation that it worked." Hermes runs the check, confirms permissions are correct, confirms the skill is enabled, confirms vision is working.

If you want to skip the dedicated CLI and just enable the tool interactively, you can do that too. Both routes end in the same place — Hermes with Computer Use enabled.

One real talk: this is an experimental feature. You're giving an agent access to your computer. If that makes you uncomfortable, don't do it. Or do it on a secondary machine. I tested it on my main machine and I'll probably uninstall later — wanted to see it run, didn't necessarily want it running all the time.

The win — Notes app journal in 8 seconds

First real test. The prompt: "Open up the Notes app and create a new note journaling about the best ways you could help me save time day-to-day."

What happened:

• Hermes used two skills in parallel — Apple Notes + macOS Computer Use
• Opened Notes from scratch (it was closed)
• Created a fresh note titled "Ways Hermes can save me time"
• Wrote ten personalised time-saving ideas grounded in my actual context — inbox management, content research, AI SEO, personal knowledge capture, second-brain surfacing
• Total elapsed time: under 8 seconds

The output wasn't generic. Because Hermes had context on what I actually do day-to-day (via the second-brain setup — see The Goldie Second Brain →), the ten ideas were specific to me. Inbox management knew which inboxes I run. Content research mentioned the AI SEO niche specifically. The system worked.

The limit — organising the whole Obsidian vault

Second test, more ambitious. The prompt: "Go into the Obsidian app and organise my knowledge base. Add details about me, structure the folders, improve the graph view — use computer use for everything."

Honest results: it struggled. Hermes opened Obsidian, started navigating, asked for permissions correctly, then got slow. The agent was rendering screenshots, parsing them, deciding where to click, and the back-and-forth slowed dramatically on a complex multi-window task.

This is the line where Computer Use isn't ready yet. For long, structural reorganisation jobs, you'll want to go a different route — use the Claude + Obsidian Second Brain setup to have Claude organise the vault via MCP directly. That's faster, cleaner, and doesn't depend on screen-reading.

Computer Use is brilliant for discrete actions — open this, create that, write a note, draft a reply, save a file. It's not yet brilliant for multi-hour structural projects. Know which job to give it.

Hermes talks to Hermes — the wild moment

The third test broke my brain a little. Prompt: "Open up Hermes in a new terminal using computer use and say hello."

Hermes navigated to my Terminal app. Opened a new window. Typed the command to start a fresh Hermes session. Waited for the prompt. Typed "hello." Hit return.

The second Hermes — running in the new terminal — replied: "Hey Julian, what are we working on today?"

Hermes literally talked to Hermes. Two copies of the same agent on the same machine, one operating the keyboard for the other.

Hermes talks to Hermes. One agent operates the keyboard, the other answers from the other side of the terminal. Reflection meets reflection.

I don't have a serious use case for this yet. But it tells you the underlying capability is real — Hermes can drive any terminal session, any app, any workflow that responds to keyboard and mouse.

The models that work — and the ones that don't

Computer Use is free, but you're routing it through an underlying model. Hermes is the harness. The model does the seeing-and-deciding.

• Claude — works well. Vision-strong, agent-tuned. The default if you have a subscription.
• OpenAI (Codex / GPT-4o) — works well. Burns a lot of tokens (screenshots are expensive). I hit my Codex usage limit during testing.
• Gemini — works. Multimodal-strong. Worth testing.
• Free Owl Alpha on OpenRouter — works for basic computer use. Free. See The Sovereign Stack →.
• Local models via LM Studio — works if the model has vision and your machine has enough memory. Some models in LM Studio's list will be marked "too large" for your hardware.
• Text-only models — do not work. The agent needs vision to see screenshots.

Token usage is the real cost factor. Screenshots eat tokens fast. There's a token-efficiency guidelines doc inside Hermes worth reading — there are optimisation layers that reduce token spend on long workflows.

The guardrails — and the limits

Hermes ships with multi-layer guardrails around Computer Use. Anything destructive — deleting files, sending emails, modifying system settings — triggers an approval prompt. You see what the agent intends to do before it does it. You can deny.

This matters because we all saw what happened when earlier agents (OpenClaw included) ran without approval gates. They sometimes did crazy stuff. The guardrails here aren't perfect, but they're meaningful.

Other current limits worth knowing:

• Mac-only. Windows and Linux not supported yet.
• Performance matters. Complex tasks on slower machines run slow. Newer Apple Silicon does better.
• Screen-reading is rate-limited. The agent waits between screenshots so you don't burn money on edge cases.
• It's experimental. Behaviour will change between versions. Treat the current capability as a snapshot.

Four beliefs holding you back

Most people will hesitate. Here's why they shouldn't.

30-day roadmap

The Goldie Shadow Worker™ — recap

Hermes Computer Use is a new free feature that lets the agent operate your Mac in the background while you keep working. The big shift isn't that it can click — earlier agents could click. The shift is that it can click without taking your keyboard away. You and the agent run in parallel on the same machine.

Brilliant for discrete tasks. Not yet ready for hours-long structural projects. Free to run. Five minutes to install. Wire it into Agent OS and it becomes one more pair of hands sitting next to every other agent in your sidebar.