Optimizing Hermes Under the Hood

A little over a month ago, I started using Hermes instead of OpenClaw. Carefully at first: a few personal tasks, some experiments, the usual “will this survive real life?” check. Then, as these things often go, it quietly became part of my workday.

Right now, a noticeable part of my personal and work tasks already goes through Hermes. I mostly use it together with Telegram: drop an idea, ask it to check something, run a small operation, come back later, and continue without starting from zero.

Lately I have been playing more with agent orchestration inside Hermes. Profiles with different models, different working modes, Kanban, separation between personal and work contexts. All of that quickly leads to one boring but important question: how much does the work Hermes does under the hood cost?

Agents Have Background Costs Too#

When we say “the agent answers with model X,” it sounds simple. But the real system does more than just produce the main answer. Hermes runs many service operations around the conversation: compressing history, extracting text from web pages, selecting skills, generating session titles, checking dangerous commands, processing images, and so on.

By default, these tasks run on the same model as the main conversation. That is convenient. But it is not always reasonable.

If the main model is expensive or tied to subscription limits, there is no point in sending all service work through it. A session title, selecting the right skill / MCP, or pulling text out of a page are simple background tasks. A smaller model is usually enough for them.

My Hermes setup runs through a Codex subscription, so the question became very practical. I want to save tokens and limits without turning the system into a lottery.

This is not the only option. The same background operations can also be routed to local models or API models if that is more convenient or cheaper. In my case, this is specifically the Codex subscription path: keep the strong model where it is actually needed, and hand the simpler internal work to a smaller model.

What I Moved to Mini#

Hermes has auxiliary models for this. The idea is simple: the main model stays responsible for conversation and complex tasks, while background operations get their own routes.

The full list of these operations is best checked in the current config.yaml: it depends on the Hermes version and enabled features. Here I am only talking about the items I changed in my setup.

I applied these settings to four of my working profiles, so service tasks behave the same way across different modes.

I moved these tasks to openai-codex / gpt-5.4-mini:

• compression — conversation history compression;
• web_extract — extracting text and meaning from web pages;
• skills_hub — selecting relevant skills;
• title_generation — generating short session titles.

These are text-heavy, frequent, and relatively low-risk tasks. A mistake there is unpleasant, but usually not catastrophic. The savings can be noticeable, especially if Hermes is not just a toy for one evening but a permanent working layer.

There is one downside to this approach: you need to keep an eye on the selected models. Providers update their lineups, rename models, and gradually move older options to deprecated status. A config that looks clean today may need a revision a couple of months later.

What I Did Not Downgrade#

I kept approval on openai-codex / gpt-5.4.

This is not the place to save pennies. approval helps decide whether a potentially dangerous command can run automatically or should ask the human first. If that check is wrong, the consequences can be more expensive than the extra tokens.

Checking the Change#

After changes like this, configs need to be checked. It is not enough to assume the command worked. I read the actual values and ran validation for each of my four profiles:

hermes -p <profile> config check

After checking the configs, restart the gateway if it is running. Otherwise Hermes will keep using the old configuration, and the new routes will not be picked up.

Only after that restart is it worth doing a live check: ask the agent directly about the current configuration, run a few simple scenarios, and make sure the expected auxiliary models are actually connected and working.

Result#

This is a small setting, but it captures how I now think about agent systems. Choosing the “main model” is not enough. You also need to know which internal operations spend its quality and limits.

A strong model belongs where there is reasoning, responsibility, and risk. For service routine, a smaller model is often enough. Not because saving money at any cost is the goal, but because different jobs should go to different tools.

In that sense, Hermes is becoming less like just a Telegram bot and more like a small operating system for tasks. And as it turns out, even an operating system has utility bills.