Hermes Agent prompt caching means your AI assistant does not pay to re-read the same long conversation from scratch every time you send it a new message.
Think of it like a receptionist who keeps the client file open on the desk instead of fetching it from the archive for every phone call.
According to the official Hermes documentation, caching trims input token costs by about 75% on chats with many turns.
Input tokens are the words the AI reads before it answers, so a long chat with your assistant gets much cheaper to keep going.
This guide covers what it is, when it works, how to set it up and what to avoid.
Why this matters for a small business
Most owners use Hermes for back-and-forth work: drafting emails, sorting leads, writing reports, checking data.
Each of those chats gets longer with every message.
Without caching, the AI provider reads the whole thing again on every reply, and you pay for that reading each time.
With caching, the old part of the chat is read from a stored copy at a much lower input cost.
If your team keeps one long working chat open all morning, this is the feature doing the saving.
For more ways owners put the assistant to work, see our guide to Hermes Agent for business.
Does it work with the AI model I use?
It switches on by itself when two things are true.
- You are using a Claude model from Anthropic.
- You connect to it through Anthropic directly or through OpenRouter.
If you use Nous Portal, Hermes checks the Nous model list to see which models should get caching.
Other providers can run their own caching, which Hermes keeps track of, but the setup described here is the Claude one.
There is no box to tick in Hermes to get it.
🔥 Want your AI assistant set up without surprise costs? Inside the AI Profit Boardroom business owners learn to run AI agents properly: plain video lessons, the Agent OS you can install, four coaching calls each week and 3,400+ members to ask. → Join the AI Profit Boardroom
How it works, without the jargon
Anthropic lets each request carry up to four "bookmarks" that say "you can reuse everything up to here".
Hermes puts the first bookmark on its standing instructions, the part that never changes.
It puts the other three on your three most recent messages, and moves them forward as the chat goes on.
The developers call this layout system_and_3, and you can read the code in prompt_caching.py on GitHub.
You never see the bookmarks; they are added to a copy of your chat just before it is sent.
Rank in AI Search or It's FREE →
How to set it up in three steps
- Update Hermes. Version 0.23.3, released on 20 November 2026, is the one with the improved cache handling.
- Open config.yaml. Look for a section called
prompt_caching. - Leave cache_ttl on auto. That is the default, so most owners change nothing.
prompt_caching:
cache_ttl: "auto"
The full list of settings is in the configuration docs.
What the three settings mean
| Setting | What it does | Good for |
|---|---|---|
| auto | Hermes picks the time window for you | Almost every business |
| 5m | Keeps the stored copy for five minutes | Fast automated jobs only |
| 1h | Keeps the stored copy for one hour | People who pause often |
On auto, a chat with a person typing gets the one-hour window.
Automated work gets five minutes: scheduled cron jobs, webhooks, helper agents, kanban tasks, API calls, batch runs and similar.
That split makes sense because a script sends its next message within seconds, while a person may step away for a coffee.
If you schedule jobs, our Hermes cron job guide shows how those run.
Join the AI Profit Boardroom →
Five things that quietly break the cache
The stored copy only helps if the start of the chat is exactly the same as last time.
- Switching AI model mid-chat. Hermes ties the stored copy to the model, so the next reply starts from zero and re-reads everything at full input cost.
- A backup model taking over. If your main model fails and Hermes falls back to another one, the cache is lost the same way.
- Rotating to another account. When Hermes moves to a different login in its key pool, that account has no stored copy.
- Editing earlier messages. Changing or removing something in the middle invalidates everything after it.
- Walking away too long. When the time window runs out, the stored copy expires.
Simple habits that keep costs down
- Choose the model at the start of the chat and keep it.
- Set up instructions and skills first, then start the real work.
- Start a fresh chat with
/newfor each new job. - Tell your team not to edit old messages in a long working chat.
A typical working day, with caching on
Picture an assistant that handles customer emails from 9 am.
The first message of the day is read in full, because nothing is stored yet.
Every reply after that reuses the stored start of the chat, so only the newest email is read at full input cost.
At lunch the chat sits idle; on auto it has the one-hour window, and warming keeps it alive for a while.
If someone switches the model at 2 pm to try a different one, the next reply starts cold and re-reads the whole morning.
That one habit change is the difference between a cheap afternoon and an expensive one.
Questions to ask whoever set up your Hermes
- Which model and provider are we using, and is it Claude?
- Is
cache_ttlstill on auto? - Do we have a backup model set, and how often does it kick in?
- Are we rotating between several accounts or keys?
- Do the scheduled jobs run in their own sessions?
Five yes-or-no answers tell you whether caching is doing its job.
What the 20 November update changed
The v0.23.3 release notes say Hermes now has one prompt-cache lifecycle controller.
- It keeps the cache warm while a long tool job is running.
- It shrinks idle chats before the one-hour window runs out.
- Nous Portal users get the full one-hour treatment.
- The setting now defaults to auto.
Four days earlier, v0.23.2 gave every chat a single cache timer.
A canary build on 19 November made sure a broken caching setting can no longer stop Hermes from starting.
What "warming" means
Warming is Hermes sending a tiny keep-alive so the stored copy does not expire while you are away.
It checks at five sixths of the time window and sends no more than three keep-alives for each message you send.
If the cache is missed for real twice, Hermes stops warming for the rest of that chat.
Rank in AI Search or It's FREE →
How to check it is working
- Type
/contextin a chat. - Read the cache line it shows.
- If it says
warming_off, the cache was missed twice in that chat. - Type
/newto start fresh, which resets warming.
Shrinking a long chat (compression) clears the cache for the shrunk part, but the instructions stay cached and normal caching returns within a turn or two.
Join the AI Profit Boardroom →
FAQ: Hermes Agent Prompt Caching for Business Owners
Do I need to switch Hermes prompt caching on?
No. If you use Claude through Anthropic or OpenRouter it is already on, and on Nous Portal the model list decides.
Why did my AI bill jump after changing models?
The stored copy belongs to one model. A switch, a fallback or a move to another account makes the next reply re-read the whole chat.
Which cache setting should a small business use?
Auto. It gives people one hour and automated jobs five minutes.
Does shrinking a long chat stop caching?
Only for the part that was shrunk. The standing instructions keep their stored copy, and savings come back after a turn or two.
How do I know if caching stopped working?
Type /context. If it shows warming_off, two real misses happened, and /new starts a fresh chat.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link building agency, Goldie Agency, with a team of 70+ SEO experts.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote "Link Building Mastery".
- My Udemy courses have taught over 29,000 students.
→ Get my best AI training inside the AI Profit Boardroom
Update Hermes, leave the setting on auto and type /context during your next long working chat.
Get step-by-step help inside the AI Profit Boardroom, and come back here whenever you need a refresher on Hermes Agent prompt caching.











